>Andy Blumenthal Talks about Cloud Computing

>Here is the podcast from MeriTalk Silverlining Series (August 2009)

>Andy’s Cloud Computing Presentation on MeriTalk

>

Introduction

First let me start out by saying that cloud computing brings us closer than ever to providing IT as a utility such as electricity, where users no longer need to know or care about how IT services are provided, and only need to know that they are reliably there, just like turning on the light. This is the subscription approach to using information technology, where base services are hosted, shared, and you pay only for what you need and use.

In cloud computing, there are a number of basic models. First, in public clouds, we have a multi-tenant, shared services environment with access provided over a secure Internet connection. In contrast in a private cloud, the IT shared services is behind the company’s firewall and is controlled by in-house staff. Then, there is also a community cloud, which is an extension of the private cloud, where IT resources are shared by several organizations that make-up a specific community.

The advantage to cloud computing—whether public or private—is that you have a shared, enterprise-wide solution that offers a number of distinct advantages:

  1. Efficiency–with cloud computing, we build once and reuse multiple times—i.e. we share resources—rather than everyone having their own.
  2. Flexibility–we are more nimble and agile when we can quickly expand or contract capacity on-demand, as needed—what some call rapid elasticity. Moreover, by outsourcing the utility computing elements of our IT infrastructure, we can focus our internal efforts on building our core mission areas.
  3. Economy (or economy of scale)–it’s cheaper and more cost effective when we can tap into larger pools of common resources maintained by companies with subject matter expertise. They then are responsible for ensuring that IT products are patched, upgraded and modernized. Moreover, we pay only for what we actually use.

Issue

So cloud computing sounds pretty good, doesn’t it? What then is the big issue? Plain and simple it comes down to—Is cloud computing effective for the organization? And what I mean by that is a few things:

  • First is customization, personalization and service: when you buy IT computing services in this shared services model, do you really get what you need and want – or are you just getting a canned approach, like the Model T that came in one color, black? For example, when you purchase Software as a Service are you getting the solution you need for your agency or the one built for someone else?
  • Next is security, privacy, and disaster recovery. This is a big deal because in a public cloud, you are capturing, processing, sending, and storing data outside of your proprietary infrastructure. This opens the door for theft, manipulation, or other ways of our data being compromised by criminals, cyber-terrorists, and even hostile nation-states.
  • Third, and maybe most important, is cultural, especially in a very individualistic society, like ours, where people are used to getting what they want, when they want, without having to share. For example, we prefer owning our own vacation home to having a time-share. We love the concept of a personal home theater. Everyone now has a personal cell phone, and the old public telephones that were once on every corner are now practically extinct. And most people prefer driving their own cars to work rather than using mass transit—even though it’s not environmentally friendly. So the idea of giving up our proprietary data centers, application systems, the control of our data, in a cloud computing model, is alien to most and possibly even frightening to many.

The Reality

So how do we harmonize the distinct advantages of cloud computing—efficiency, flexibility, and economy—with the issues of customization, security, and culture?

The reality is that regardless of customization issues, we can simply no longer afford for everyone to have their own IT platforms—it’s wasteful. We are recovering from a deep financial recession, the nation has accumulated unprecedented levels of debt, and we are competing in a vast global economy, where others are constantly raising the bar—working faster, better, and cheaper.

Moreover, from a technology standpoint, we have advanced to where it is now possible to build an efficient cloud computing environment using distributed architecture, virtualization/consolidation, and grid computing.

Thirdly, on a cultural level, as individualistic as we are, it is also true that we now recognize the importance of information sharing and collaboration. We are well aware of the fact that we need to break the stovepiped verticals and build and work horizontally. This is exemplified by things like Google Docs, SharePoint, Wikipedia, and more.

In terms of security, I certainly understand people’s concern and it is real. However, we are all already using the cloud. Are you using online banking? Are you ordering things online through Amazon, Overstock or other e-commerce vendors? Do you use yahoo or Google email? Then you are already using the cloud and for most of us, we don’t even realize it. The bottom line on security is that every agency has to decide for itself in terms of its mission and ability to mitigate any risks.

How to Choose

So there are two questions then. Assuming—and I emphasize assuming—that we can solve the security issues with a “Trusted Cloud” that is certified and accredited, can we get over the anxiety of moving towards cloud computing as the new standard? I believe that since the use case—for flexibility, economy, and efficiency—is so compelling, that the answer is going to be a resounding yes.

The next question is, once we accept the need for a cloud computing environment, how do we filter our choices among the many available?

Of course I’m not going to recommend any particular vendor or solution, but what I will do is advocate for using enterprise architecture and sound IT governance as the framework for the decision process.

For too many years, we based our decisions on gut, intuition, politics, and subjective management whim, which is why statistics show that more than 82% of IT projects are failing or seriously challenged.

While a full discussion of the EA and governance process is outside the scope of this talk, I do want to point out that to appropriately evaluate our cloud computing options, we must use a strong framework of architecture planning and capital planning and investment control to ensure the strategic alignment, technical compliance, return on investment, and risk mitigation—including of course security and privacy—necessary for successful implementation.

How Cloud Computing fits with Enterprise Architecture:

As we move to cloud computing, we need to recognize that this is not something completely new, but rather an extension of Service Oriented Architecture (SOA) where there are service providers and consumers and applications are built by assembling reusable, shared services that are made available to consumers to search, access, and utilize. Only now with public cloud computing, we are sharing services beyond the enterprise and to include applications, data, and infrastructure.

In terms of a transition strategy, cloud computing is a natural evolution in IT service provision.

At first, we did everything in-house, ourselves—with our own employees, equipment, and facilities. This was generally very expensive in terms of finding and maintaining employees with the right skill sets, and developing and maintaining all our own systems and technology infrastructure, securing it, patching it, upgrading it, and so on.

So then came the hiring of contractors to support our in-house staff; this helped alleviate some of the hiring and training issues on the organization. But it wasn’t enough to make us cost-efficient, especially since we were still managing all our own systems and technologies for our organization, as a stovepipe.

Next, we moved to a managed services model, where we out-sourced vast chunks of our IT—from our helpdesk to desktop support, from data centers to applications development, and even to security and more.

Finally, the realization has emerged that we do not need to provide IT services either with our own or contracted staff, but rather we can rely on IT cloud providers who can offer an array of IT services, on demand, and who will manage our information technology and that of tens, hundreds, and thousands of others and provide it seamlessly over the Internet, so that we all benefit from a more scalable and unified service provision model.

Of course, from a target architecture perspective, cloud computing really hits the mark, because it provides for many of the inherent architecture principles that we are looking to implement, such as: services interoperability and component reuse, and technology standardization, simplification, and cost-efficiency. And on top of all that—using services on a subscription or metered basis is convenient for the end-user.

Just one last thing I would like to point out is that sound enterprise architecture and governance must be user-centric. That means that we only build decision products that are valuable and actionable to our users—no more ivory tower efforts or developing shelfware. We need to get the right information to the right decision makers to get the mission accomplished with the best, most agile and economical support framework available.

>SOA Liberates Productivity

>

Harvard Business Review (HBR), June 2008, has a wonderful article (by Ric Merrifield, Jack Calhoun, and Dennis Stevens) on how SOA is “the next revolution in productivity.”

SOA defined:

“It is becoming possible to design many business activities as Lego-like software components that can be easily put together and taken apart…service-oriented architecture [is] a relatively new way of designing and deploying the software that supports a business activity.”

With SOA, business activities can be accessed via the Internet through web services. Rather than build proprietary, redundant business services, our organizations can re-use standardized services, developed internally or outsourced, as components that plug and play into our enterprise.

“Virtually all large companies suffer from an enormous duplication of activities; they continue to create and perform hundreds of non-core tasks that would ideally be outsourced; and they are spending exorbitant amounts on IT projects in order to support redundant and nonstrategic operations and to update core processes.”

How does this differ from other quality improvement initiatives?

Prior quality improvement efforts like Total Quality Management (TQM) and Six Sigma have focused on reducing waste and defects and eliminating unnecessary tasks and integrating disparate ones.

“For the most part, however, reengineering has involved recasting processes and the information systems that support them in a proprietary, rather than a standardized, form—that is, customized for individual organizations. Such designs make it difficult and expensive for business to share, consolidate, and change processes.”

Now with the Internet and web services, we can access standardized services that can be shared and re-used throughout disparate business units in the same enterprise and across organizations globally.

The result is business units and organizations that can simply plug and play to make use of needed services, eliminating proprietary processes and redundant systems and enabling outsourcing of noncore mission functions and activities and easier upgrades to new superior services as they come online.

What are some of the issues holding SOA back?

Firstly, many people do not truly understand SOA, what it is, what benefits are possible, and what the challenges are to doing it right.

Second, SOA is viewed by many executives as yet another hype or bubble that will cost the enterprise lots of money, but fail to provide the promised return. So, they are wading into SOA only enough to “deploy it in a limited fashion,” but without first rethinking the design of their business.” However, to really reap the benefits of SOA, organizations need to transform from “collections of proprietary operations into a collection of standard plug-and-play activities,” and this requires redesigning not only IT systems, but operations.

In designing SOA-based processes, the unit of analysis and reengineering is no longer the task (as in Frederick Taylor time and motion studies of the late nineteenth century) or the department, or even the division. “In the age of the Internet and SOA, the unit of analysis is not a company’s way of conducting its operations at all; it is the primary purpose or desired outcome of each activity no matter how that activity is accomplished.”

Where are we today with SOA implementation?

“Unfortunately, few companies are using SOA to create more productive and focused organizations or to slash costs by purging duplicative operations and technologies. They are not revisiting the fundamental design of their operations.”

To overcome the obstacles in reaching SOA enabled organizations, we need a strong dose of enterprise architecture to identify and decompose our performance outcomes we are driving toward, the business processes to achieve these, the information required to perform these, and the systems that can serve them up.

According to HBR, our business model activities can be categorized into the following for SOA implementation:

  • Primary (I would call this core mission)—those that should be kept in-house and are “top priority of programs to improve operations and technology” (i.e. through business process improvement, reengineering, and the introduction of new technology).
  • Shared—those that “can be shared with other divisions” (i.e. through common solutions).
  • Shifted—those that “can be transferred to customers, suppliers, or operational specialists” (i.e. outsourced).
  • Automated—those that can be “automated so they can be turned into web services.”

All but the primary activities are ripe for SOA-based enhancements. And according to HBR, only about 20% of activities are primary, so that leaves plenty of room for a SOA plug-and-play.

The idea is to cease defining our noncore mission processes and activities as proprietary, requiring elaborate and expensive customized solutions for these. Instead, we should use standardized “swapped, bought, or sold” services. Then, we can truly focus our business process reengineering and IT investments on our organization’s core mission activities—working to to differentiate ourselves and develop unsurpassed competitive advantage.

>SOA, Data Management and Enterprise Architecture

>Often I hear business and IT people say that Service Oriented Architecture (SOA) is the way ahead to achieve greater interoperability of systems, process integration, and business efficiency in the enterprise. Usually, this is quickly followed by discussion about rolling out the Enterprise Service Bus (ESB). However, very infrequently do I hear discussion about the necessary data architecture to achieve the vision of SOA.

Stephen Lahanas has an interesting article, “Enable SOA Transformation and Cross-Domain Data Fusion” in DM Review Magazine, January 2008 that addresses the importance of Data Management to SOA. (http://www.dmreview.com/issues/2007_43/10000444-1.html)

“While SOA has long considered universal description, discovery and integration (UDDI) as its primary discoverability mechanism, the reality is that nearly all integration with an SOA environment is based upon data exchange and will ultimately be demonstrated through data exploitation interfaces (agile BI). Once SOA architects fully realize the implications of this revelation, then agile data architecture will become the facilitating mechanism for cross-domain data fusion and enterprise integration.

Mr. Lahanas provides four fundamental elements for building an agile data architecture, as follows:

  1. Actionable enterprise architecture—“a tangible way to connect architecture layers (EA, segment and implementation) and perspectives (application, process, data). One of the main reasons that large integration projects fail is due to the inability to successfully map the various architectures within a meaningful combined picture. Every agile data architecture begins here.”
  2. Federated data orchestration—“allow data owners to collaboratively manage resources across domains based upon a shared set of rules rather than a shared single data model. This is an excellent example of a user-centric approach. One of the major shortcomings of massive data warehouse projects has been the lost connections between users and developers and resulting data integrity issues.
  3. Enterprise Master Data Management—“Metadata is not just a technical consideration; it can define productivity in our knowledge economy.”
  4. Agile Business Intelligence—“A new generation of BI capabilities is bringing user control to more sophisticated report generation tools with much more accurate results… Based upon user queries and activities, we gather metadata, optimize caches and determine cross-domain mapping strategies.”

Two ideas that I particularly like in Stephen’s article are:

  1. Linkage of SOA to Data Architecture—“Agile data architecture is a parallel and complementary design philosophy and methodology to SOA and agile application development. Both can be mapped together within the larger actionable enterprise architecture.”
  2. Focus on User-centricity—“User-centricity is the primary motivating force behind the development of all agile solutions. The user provides:
  • Immediacy – the desire for near-term real-world capability.
  • Relevance and context.
  • Performance expectations.
  • Direction, domain knowledge and the logic behind every solution.”

In developing SOA, we cannot forget the necessity of building a meaningful data architecture that facilitates the discovery and exchange of data via SOA and ESB. Further, all our architecture and IT solutions must be user-centric if they are to be relevant and effective to the enterprise’s end-users.

>SOA – Does the Emperor Have Clothes?

>

Architecture and Governance Magazine, Volume 3 Issue 3, reports that one of the Top 10 Challenges in Enterprise Architecture is “Getting Ready for Service-Oriented Architecture (SOA)”.

– Why is SOA proving to be such a challenge?

The article reports that “most architects have not done much of either substance or significance in this area.” Further, “most of what enterprise architects have addressed in this space concerns aligning IT with the business and governance”—and this is something that EA has been doing all along anyway.

– So what has SOA changed?

“What has changed is the promise of abstracting services that can be reused.”

However, some have questioned SOA:

  1. Is SOA “just a revival of modular programming (1970s), event-oriented design (1980s), or interface/component-based design (1990s)”?
  2. Is SOA “just another term for Web Services”?
  3. Is SOA “merely an obvious evolution of currently well-deployed architecture (open interfaces, etc.)”? (Wikipedia)

Others have questioned if SOA and EA are even distinct disciplines?

“So, has SOA in fact evolved into EA? A to question to which the answer is yes and no.” The author continues, “essentially there are three evolutionary paths:

  1. EA is absorbed by SOA
  2. SOA is absorbed by EA
  3. EA / SOA merge into a new concept”

Finally, the authors propose #3 merging EA and SOA as follows:

“SOA-A+EA = SOEA”

With the conclusion that “in order to create this holistic view EA and SOA must not be seen as separate disciplines, but to see the two as one will bring changes on such a scale that we are now talking about SOEA Service Oriented Enterprise Architecture.” (Journal of Enterprise Architecture, August 2007, Rasmus Knippel and Bo Skytte)

While the authors apparently seems to think SOA is important (although they call it “increasingly more abstract”), the relationship to EA is changing and unclear—Not sure this idea about SOEA is helpful at this juncture!

– How helpful to SOA development is the guidance from the Federal Enterprise Architecture (FEA), Service Reference Model (SRM), which talks about services and capabilities?

Not very! The SRM is supposed to “provide a high-level view of the services and capabilities that support enterprise and organizational process and applications.” (FEA, SRM). However, the service domains, service types, and capabilities listed in the FEA SRM are quite muddy, at best. Here is an example:

  • The domain of Process Automation is defined as “the set of capabilities that support the automation of process” [hasn’t every 5th grader been taught not to define a term with the same term?]
  • And it continues “and management that assists in effectively managing the business” [well that is a pretty broad definition— can’t all services assist in effectively managing the business?].
  • Then after the definition, you’re expecting some pretty extensive process automation services, but the SRM lists just 2 components:
  1. Tracking and Workflow (which includes capabilities like case management [what’s that doing here?] and conflict resolution [isn’t that a leadership skill set?])
  2. Routing and Scheduling, which includes just Inbound and Outbound Correspondence Management [so where the scheduling piece like calendaring or time management that one normally associates with scheduling and how does the routing as in Routing and Scheduling reflect the routing inherent in Tracking and Workflow category above—seems like some category overlap]

I’ll let you look at the SRM yourself to see evaluate the clarity of the other categories, but they are pretty much the same. Ok, I can’t resist, here’s one more:

  • The domain called Business Management Services, after the domain of Process Automation, actually has a service type in it called management of process {why isn’t this in the process automation domain?].
  • The other service types in this domain include Organizational Management [that includes workgroup/groupware and network management, huh?]
  • And Investment Management [ok]
  • And Supply Chain Management [which includes things like procurement, Warehousing, and Inventory Management— so why isn’t this part of back office services, like Finance, Human Capital, and Asset/Material Management?]

If this looks confusing, you’ve got my point, it’s because the FEA SRM is confusing.

– So what’s an enterprise architect to do?

Architecture and Governance Magazine says, enterprise architects need to take back the high ground here—and that must be done in 2007…today’s enterprise architect must re-engage with all the principals in the enterprise by making a case for seeing service and technology architectures as one.”— this is good advice, I suppose, but also more than a little vague!

I’ve seen a bunch of presentations (including a recent one from Department of Education) where EA has tied IT services (pretty much their IT portfolio) to requested business capabilities—again it seems pretty much like what EA is or should have been doing all along, aligning business and technology investments with a focus on reducing gaps, redundancies, and inefficiencies.

I’ve seen other people including some prominent computer and consulting vendors present this topic as well, and aside from pushing there wares, have seen them fall all over themselves tying to define and explain SOA. Honestly, it was pathetic!

Finally, I have seen one impressive implementation at a government agency, and it did actually have savings associated with it, but the application was done a few years back, was relatively small, and there has been little to no movement on further SOA development since (except for the purchase of an enterprise service bus) even though there is a mandate in this agency to implement SOA.

In fact, at a recent meeting with the business side from operations, the presenters said that they were going to spend oodles of money rewriting one (if not the) major operational application for the agency (over 1.2 million lines of code and growing) to make it .NET compliant. I asked the business if they were going to take this opportunity to do this using SOA, and they said no, they just want the same functionality as before!

So in the end, if the business doesn’t want it and the vendors and consultants can’t explain it, the final question with SOA is does the emperor really have clothes?

>SOA Best Practices

>I came across these Service-Oriented Architecture (SOA) best practices and thought they aligned well with User-centric EA and were worth sharing:

(adapted from Naval Post Graduate School, Monterey, CA, Masters Thesis—SOA for Coast Guard Command and Control by Dash and Creigh, March 2007 — approved for public release):

  • Know when to use services – Explicitly define the extent to which we will and will not use services; services are selected with intent and do not randomly spring up.
  • Think big, start small – This allows us to validate the architecture, while giving the organization value, realized as usable services.
  • Build on what you have – Reuse legacy application functionality and data wherever possible.
  • Use SOA to streamline business processes – SOA is an inherently flexible and interoperable model for hosting application functionality; this provides an opportunity to rethink and improve business processes.
  • Incorporate standards – Use industry web service standards for navigation, application logic, integration, data stores, and enterprise infrastructure.
  • Build around a security model – Functional design needs to be built around security, and not vice versa (security cannot be added as an afterthought).
  • Design with quality in mind – Quality must be designed into the product, and not inspected into it.
  • Organize development resources – Group development team around logical business tasks, and not around technologies.
  • Train developers – Ensure designers and developers have the skills to implement SOA properly.

Basic definitions:

SOA is a software design methodology that uses loosely coupled services to perform business functions and processes.

A service is an implementation of a well-defined piece of business functionality, with a published interface that is discoverable and can be used by service consumers when building different applications and business processes.

In general, I think SOA is a great fit with User-centric EA, because the focus is on providing functional business services to the end-user, as opposed to developing monolithic, stove-piped, and redundant applications. The end-user should not have to use countless non-integrated applications systems (or even computers) to get their information, but rather the information should be technically transparent and readily accessible based on their business requirements.