The only Eucalyptus I see in my travels

I saw a fantastic article from Nancy Gohring of InfoWorld yesterday, on how “Amazon said that it would back Eucalyptus’ efforts to support Amazon Web Services’ APIs”. Great article, well worth reading in full.

For me, however, it was the a priori assumption in the first paragraph (and the headline) that really stood out:

Eucalyptus has become far more attractive to enterprises wishing to build private clouds, now that the No. 1 cloud provider — Amazon Web Services — has thrown its weight behind the software company.

I am not buying this at all.

In my experience with many enterprises actively moving to the cloud, most every large organization sees Amazon Web Services (AWS) as an aspiration, but not a preference. They tell me that they want to be like AWS, but typically only use AWS for edge cases and new developments – and typically non-mission critical applications, rather than mainstream production. At least for now. (I cannot comment on attitudes to Eucalyptus – I do not know any enterprise that is considering it.)

What’s more, in a separate (and also excellent) article on GigaOm, comments from a Public Relations Manager for Amazon suggest to me that the world’s largest cloud provider still doesn’t really ‘get’ the reality of enterprise computing:

“Many enterprises today have legacy applications and a good deal of investment in those legacy applications. This type of arrangement provides the added flexibility to more freely move workloads between those existing IT environments and AWS … [O]ver time, most enterprises will not run their own data centers.”

This thinking sounds great as a PR statement, but in the real world it discounts the clear intentions of almost every large enterprise, and contradicts almost all research data. Unless “over time” means on a geological scale, the opposite is actually true.

Amazon simply does not support the multitude of platforms, systems, and vendors that are typical of “legacy applications” in large enterprises. For many, it does not accommodate a lot of mandatory requirements for management, security, compliance, etc.

In a new article published yesterday in CIO.com, the inestimable Bernard Golden notes that for many enterprises, moving ‘legacy applications’ to a cloud environment is not practically possibly:

Cloud computing is not going to solve legacy application challenges and costs. I recently talked with the CIO of a large media company who commissioned a study of his legacy apps to determine how many could operate in a cloud environment. The results: 10 percent.

At very least, Bernard continues, IT would need to completely re-engineer most legacy applications for AWS. However, in my experience very few enterprises see a compelling need to re-engineer millions of lines of legacy code that still does what it needs to. The other realistic option for legacy applications, Bernard points out, is “shifting to on-demand SaaS applications”, not migrating to an IaaS provider.

Which leaves AWS out in the cold when it comes to this supposed movement of legacy applications.

Meanwhile, even the poster child for AWS migration, Netflix, still has a significant legacy IT environment; and cloud-native developer Zynga is actually moving non-legacy applications off AWS.

So then, how then does “this type of arrangement” allow a large global bank to “more freely move” its SWIFT processing onto an AWS cloud? How can a mineral exploration company “more freely move” to AWS for geological data processing from a remote oil field in Azerbaijan? How can a research institute “more freely move” to a hybrid AWS cloud to run trillions of calculations a second to model 100 years of global climate change? Or any of the other thousands (millions?) of legitimate, actual, legacy (and current) enterprise use cases for non-commodity environments?

I don’t think it does.

Even for new applications, most enterprises are not showing a preference for actually using AWS; and Eucalyptus is still being roundly trounced in the market for enterprise private cloud.

This is not to say that AWS is not a viable choice for enterprise adoption. It certainly is capable of running large-scale enterprise applications in production, so long as they are engineered for the environment, and supported by capable third-party tools for security, orchestration, assurance, service levels, and so on. At CA Technologies I am working every day with our enterprise customers, encouraging them and helping them with public cloud (including AWS) adoption.

However, I do not believe that Amazon’s new alliance with Eucalyptus clears any significant barriers for enterprise adoption of public or private cloud. Enterprises that were adopting AWS for certain use cases will continue to do so; presumably any enterprise that is building a Eucalyptus private cloud will continue to do so.

But neither makes the other any more ‘enterprise-ready’ than it already was.

*btw, bad Australian joke for you: What does a Koala do at a party? Eats roots and leaves! Yeah, the Aussies Antipodeans will get it.

How does open cloud stack up on price?

After Australia’s Melbourne IT unceremoniously dumped VMware vCloud Express, I wondered whether proprietary offerings like vCloud Express can provide the margin to compete with equivalent open source cloud offerings (e.g. based on Xen or KVM).

I am not alone either. Respected analyst Dale Vile (co-founder of Freeform Dynamics) posed similar questions on Twitter.

In response, one VMware employee posted a Virtacore vCloud Express price list and asserted that vCloud Express pricing is “very competitive with any cloud” and that the cost of proprietary cloud was a “non-issue”.

However, it turns out that the ‘non-issue’ status of vCloud Express pricing is far from universal, even within VMware. At the recent Cloud Expo event (my roundup and slides for my three sessions are here), another VMware employee seemed to think it an issue worth addressing, as he chose to highlight the same pricelist in his presentation as ‘proof’ that proprietary cloud was actually more cost effective than a public cloud offering.

Moreover, one pricelist in isolation says nothing about competitive pricing; rather it simply demands comparison.

So spurred on by this broad concern and spirited commentary, I decided to compare the published list prices of multiple proprietary offerings (including the now-defunct Melbourne IT vCloud Express price list) with roughly equivalent offerings (based on open source Xen) from Amazon and Rackspace.
This turned up some confusing results that, for the most part, do not seem to support the VMware position. For example:

• No proprietary offering comes close to matching the price of the cheapest open offering, AWS’s ‘Free Tier’
• Melbourne IT’s cheapest offering was over twice as expensive as the lowest paid-for offering from Amazon, which offers the same CPU, but four times the RAM
• Melbourne IT’s cheapest offering was almost five times the cost of the cheapest Rackspace service, which offers (up to) four times the CPU power, albeit only half the RAM
• Every one of Amazon’s offerings are cheaper than proprietary equivalents, except for the 4 CPU, 2 GB server service
• Every one of Rackspace’s offerings are cheaper than proprietary equivalents, except for the (up to) 4 CPU, 16 GB server service
• For a relatively standard server (1 CPU, 2Gb RAM), Amazon is half the cost of Melbourne IT, and two-thirds the cost of the other equivalent proprietary offerings
• For a high-powered server (4CPU, 8Gb), an open offering from Rackspace is around the same price as two of the three equivalent proprietary offerings
• At the most expensive price point ($0.96/hr), a proprietary offering beats Rackspace for the same memory while providing twice the CPU, but is still more expensive than an equivalent Amazon service ($0.68/hr)

Perhaps this helps to explain why the CTO of Melbourne IT, Glenn Gore, is reported as saying “the market for [SMB] customers was already dominated by large scale American offerings such as Amazon Web Services and Rackspace”.

Of course, list price should never be the sole factor in deciding the value of a cloud service. Many other factors should come into play, including reliability, security, manageability, uptime, integration, portability, support, and much more. Nor is this a clear apples-to-apples comparison. Each service has differences in specific offerings, few services compare equally head-to-head, and each provider has different cost structures for bandwidth and storage. Amazon and Rackspace in particular obfuscate the CPU basis for their offerings, making any comparison difficult at best, flawed at worst.

In the interest of open discussion then, I have included my full data set below, along with some additional explanatory pricing notes and links to the original sources. Maybe I have interpreted some of the sources incorrectly. Maybe there are additional values I have not accommodated. Most notably, my conversion of Amazon’s “Compute Units” or Rackspace’s variable CPU allocations may be flawed (they do not make it easy). While I think this data does enable a valid comparison, in the end it is just my interpretation of some very dissimilar price lists.

I would love to know what you think. Can a proprietary cloud offering compete on price with an offering based on open source? It appears to me that this is the exception, rather than the rule, but VMWare keep insisting that it is true.

So what am I missing?

Please post any comments and corrections below – I would love to figure this one out.

Pricing Notes

Melbourne IT

• As noted, Melbourne IT is no longer offering a vCloud Express service
• Storage is additional $0.327 Per GB per month
• Bandwidth is additional $1.047 per GB
• Prices are all in AUD. At current exchange rate these prices are slightly higher in USD

Source:

http://vcloudexpress.melbourneit.com.au/

Virtacore

• Unused instances are charged at full rate, even if unused, unless ‘destroyed’
• Bandwidth is free up to 3TB, although the time period for this limit is not clear
• All plans include 50 GB Disk
• Plans are actually listed as providing ‘VCPU’

Source:

http://www.virtacore.com/vcloud_pricing.cfm

Terremark

• System Storage $0.25/month per GB (billed with virtual machine)
• Internet Bandwidth $0.17 per transferred GB
• Public IP Addresses $0.01/hour per IP
• Internet Services $0.01/hour per service
• 8 VCPU also available

Source:

http://vcloudexpress.terremark.com/pricing.aspx

Hosting.com

• System Storage $0.50/month per GB (billed monthly)
• Bandwidth $0.17/GB (billed hourly)
• It is unclear if Hosting.com continues to offer vCloud Express. The vCloud Express web page does not appear to be accessible by navigating the menu system (only options are ‘Enterprise’, ‘Dedicated’, and ‘Private’), and the service is still classed only as ‘beta’, but the page can be found by searching the website.

Source:

http://www.hosting.com/services/platform-services/cloud/vcloud-express

Amazon Web Services (EC2)

• AWS actually leases by EC2 Compute Units (virtual cores with 1 or 2 ‘Compute Units’, each providing “equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor”). I have converted these at a 1:1 ratio, but this obfuscation does make it difficult to compare on even footing with other providers.
• All data transfer in – $0.100 per GB; Data Transfer out – First 1 GB / month free, and a reducing scale after that at an average of $1.07 per GB / month
• Offerings listed as 0.5 GB RAM actually provide 613MB, not 512MB; Offerings listed as 2GB RAM actually provide 1.7GB RAM; Offerings listed as 8GB RAM actually provide 7.5GB RAM; Offerings listed as 16GB RAM actually provide 15GB RAM. These have been rounded for ease of comparison.
• Free Tier is only available for 12 months for new customers

Sources:

http://aws.amazon.com/ec2/pricing/

http://aws.amazon.com/ec2/instance-types/

http://aws.amazon.com/free/

Rackspace

• Bandwidth costs: Out 18¢ / GB; In 8¢ / GB
• Rackspace’s CPU allocation methodology is not transparent, and may not be as interpreted. For Linux distributions, it appears to offer up to four virtual CPUs for every Rackspace Cloud Server. However, CPU measurement is based on a non-standard and dynamic weighting that is far from clear. This makes it difficult to compare 1:1 with other providers, even after consulting with Rackspace support.

Sources:

http://www.rackspace.com/cloud/cloud_hosting_products/servers/technology/

http://www.rackspace.com/cloud/cloud_hosting_products/servers/pricing/

Which is the bigger myth - public cloud cost savings or Lewis Carroll's Jabberwock?

There is a lot of hype around how much money you can save with public cloud.

Public cloud can be cheaper than on-premise IT or private cloud, especially for selected services and SMBs. However for large enterprises, while there are plenty of reasons to use public cloud, cost reduction is not always one of them.

Public cloud certainly has a low startup cost, but also a long ongoing cost. For all practical purposes, the ongoing cost is never-ending too. As long as you need it, you keep paying as much as you did on day one, without adding an asset to your books or depreciating your facilities investments.

Public cloud also rarely (ever?) provides the management, security, governance, or lifecycle solutions that add cost to internal IT either – not because public cloud does not need them, but because it eats into cloud providers’ profit margins. Indeed, the myth of low cost cloud is built directly on the equally false myth that public cloud doesn’t need management or discipline.

Better economies of scale are not assured either. Many large IT organizations have more and larger data centers than most, though perhaps not all, public cloud providers. Enterprise IT can be strikingly massive, with economies of scale that match or exceed even the largest public cloud providers.

Public cloud does not always lead to a substantial reduction in staff costs either. Simply sourcing infrastructure or software as a service does not let you disband your IT department entirely. After all, your databases still need DBAs; your applications still need performance monitoring; your users still need a Help Desk; regardless of where the services or systems are running.

This last issue was partly the basis for an early debunking of the ‘public cloud is cheaper’ myth in a McKinsey report titled ‘Clearing the Air on Cloud Computing’ (PDF) back in 2009. McKinsey found that IT labor cost savings with public cloud (IaaS in particular) are minimal, around 15% mainly in sysadmins. However, non-labor costs were actually much higher for public cloud. Overall, it showed a 144% higher cost of public cloud over internal IT, and concluded that, with the majority of cloud savings achievable in-house with virtualization, an investment in private infrastructure makes sense for larger organizations.

A lot has changed since 2009, and both platform and software clouds offer different benefits to infrastructure clouds. Still, many of the CIOs that my co-authors and I interviewed for our recent book ‘Visible Ops – Private Cloud: From Virtualization to Private Cloud in 4 Practical Steps‘ confirmed this from their own experience. These practitioners have done the calculations on their own real world IT budgets, and told us how they can deliver the same (or better) service internally at a 30% discount to public alternatives – in accordance with the Visible Ops – Private Cloud mantra of ‘faster, better, cheaper’:

For example, the larger organizations we interviewed are confident that they can offer their own public cloud-like services cheaper, better, and more securely than external service providers. As a result, many are working to create a self-service, low-touch model that can be used to spin up new computing resources.

In Visible Ops – Private Cloud, we also cited primary research conducted and published by Wikibon on the comparative cost of public cloud for large enterprises. It has a more detailed analysis of economies of scale, and found that only for low-priority applications in SMB environments will public cloud be more cost-effective than private cloud, and then by only 14%. By contrast, private cloud is more cost-effective for all large enterprise use cases, and for all mission-critical applications (even in SMBs), by up to 41% annually, primarily because:

1) The applications running in larger organizations demand more resilience, governance and control;

2) While small businesses will find public clouds attractive, large organizations will be able to create a private internal cloud that has many of the same business characteristics as a public cloud, but with much higher levels of control, security and availability; and

3) Achieving the same levels of resilience with public cloud infrastructure would be prohibitively expensive for larger organizations.

ASG has also published a short analysis of the comparative TCO of private and public clouds on their blog. This analysis has many built-in assumptions, but it does show a reasonable and defensible interpretation of costs in both environments. According to this analysis, over a 3-year period, private cloud operation (whether on owned or leased equipment) is around half the cost of public cloud:

This analysis shows a $513,295 cumulative cost savings over three years for the private cloud purchase versus the public cloud option [$1,092,188]. The private cloud lease saved $489,874 versus the public cloud option over the same three years.

Then there is James Staten of Forrester research, and a great report titled ‘The 3 Stages of Cloud Economics’. Unfortunately for many, the full report is subscriber-only, so I cannot extract it, but it is a great read (you can get a free taste over at James’ blog at Forrester.com on the stages of cloud economics). Suffice to say that the report starts out front and center with a whole section titled ‘BUSTING THE MYTH — CLOUD COMPUTING ALWAYS SAVES YOU MONEY’ [sic].

Of course, a realistic cost analysis depends on too many organization-specific factors to draw firm conclusions for any but specific use cases. However, for most enterprises evaluating cloud (public or private), an overly tight focus on cost reduction is misplaced at best. Given all the experience and data at hand, cost reduction is at best a possible outcome from public cloud, and even then only for some workloads, and only for some organizations.

Yet all of this ultimately misses the point. As James Staten says in his blog:

But this isn’t a debate worth having because it’s the exploration of the use cases where it does save you money that bares the real fruit. And it’s through this experience that you can start shifting your thinking from cost savings to revenue opportunities.

Clearly cost reduction is important when you can achieve it. Indeed, judicious use of cloud computing, public and private, for the right services, the right users, at the right time, certainly does deliver cost reductions. However, while cloud in general and public cloud in particular can save money, cost reduction is certainly not the only reason to use cloud, public or private; in my opinion, it is probably not even the best reason to use cloud.

The potential that cloud computing creates to drive technology flexibility, business agility, quality of experience, and service continuity (even despite multiple high-profile public cloud outages) far exceed the returns to large enterprises of hardware cost reduction. The real benefit is found in business gains like greater competitive advantage, faster time to market, and higher customer retention.

It is through these business factors – not mundane IT line items such as server purchase prices and administrator salaries – that cloud computing, both public and private, will drive the most significant and fundamental changes to the bottom line for most large enterprises.

eWeek.com

In the aftermath of the recent Amazon EC2 outage, I spoke with Chris Preimesberger at eWeek about what this means to the perceptions of cloud computing, and whether or not it really mattered:

“The first thing to understand [about this event] is that this changes nothing,” Andi Mann, longtime storage industry analyst who’s currently serving as chief cloud strategy guru at CA Technologies, told eWEEK.

“Cloud will have downtime—it’s a fundamental issue. But you need to be ready for downtime, whether it’s your own infrastructure or cloud infrastructure. You need to understand what the risk is. It’s all just about risk management.”

You can read the whole article here – Will Amazon EC2 Outage Negatively Affect Attitudes Toward Cloud? Nah.

The author, Andi Mann (left), speaking with Roger Green of CIOZone.com during VMworld 2010

At VMworld 2010, I had the great pleasure to record a video interview with Roger Green, Executive Editor at CIOZone.com. We chatted for about 20 minutes in total (in 2 parts) about virtualization, the issues of virtual stall (including both causes and solutions), how the antecedents of virtualization can inform our modern approaches, the importance of data center automation, the impending tsunami of cloud computing, and much more.

If you have not seen it, you can find Part 1 here, and Part 2 here.

(btw, if you have not seen the video archive on CIOZone.com, you really should – it includes some fantastic interviews with many virtualization and cloud experts and thought leaders including Microsoft Director David Greschler, Rackspace CTO John Engates, VMware GM Jim Morrisroe, VMware CIO Mark Egan, and my colleague and counterpart in our Virtualization Product Management team, CA Technologies VP Subo Guha – plus a host of other CIOs and IT experts. Definitely worth your time!)

I found a comment on the CIOZone.com discussion thread about the interview very interesting. It came from CIOZone member, Pete Simmeron (petesim), who posted:

Andi Mann, thanks for the great conversation, I learned a fair amount about some of the obstacles to implementing a virtualization solution. You mention issues like license control, deprovisioning excess virtualized servers, compliance, and how IT support staff do not necessarily scale to help bring an organization to the Utopian 100% virtualized environment. Well my question is then how do we move past these obstacles? Do we start slow and develop the necessary skills in house or do we hire from outside, and thinking ahead do you think this will become a highly sought after job skill in the next 10 years??

What a great question! It inspired me to a longer response than might be appropriate in a comment box, but here it is in full:

Pete, thanks for the comment.

Moving past the obstacles is certainly the goal, and it is not easy, but it is possible. I think automation tools are massively important, but people and process are key too.

For example, to manage licences properly, and to track which VMs are being used and which can be reclaimed, some type of asset and inventory registry is an excellent tool. However, to really make this valuable, instead of just adding work manually recording VM allocations and movements, it really has to be updated automatically when VMs are provisioned and deprovisioned. This will save resources (admins don’t have to manually assign and collect VMs and licenses), save costs (licences can be more easily reused instead of adding new license costs), and help expand virtualization deployment and maturity (admins can move onto virtualizing more complex systems, rather than just babysitting existing VMs).

However, it is also important to address the people & process issues, not just the technology issues. For example, you have to make sure that IT admins and even VM owners don’t deliberately find ‘workarounds’ for the asset registry that might make their job a little easier, and maybe even save their department some money, but which actually end up costing the wider company a lot more in terms of compliance breaches and increased license costs. It is also critical to make sure there is a clear, known, and easily accessed process in place (ideally one which is automatically enforced) to work with VM owners/requestors to identify and deprovision VMs that are no longer in use. This will make sure the technology benefits accrue as expected, but also will simplify some complex and problematic VM management activities.

As to where you start, a lot depends on existing process and technology maturity, organization size, the most pressing problems, and the overall goals. Most orgs will do best to solve one problem at a time with the people they have – perhaps managing VM performance, or controlling licenses, or automating provisioning. But very mature orgs will likely be able to do more up front, like implementing a service catalog and service desk approach to automated provisioning and deprovisioning, or even combining this with resource pools and self-service to start on the journey to cloud computing. Meanwhile, smaller orgs will probably need to bring in experts at least temporarily to help them get over the hump, as they are typically harder-pressed to skill up and resource in-house for such significant IT changes.

Finally, yes, I absolutely believe this will be an in-demand skill for years to come. The evidence is already there, in the higher average pay rates for administrators with virtualization certifications, and I think this will continue. Administrators and managers that can effectively harness tools, processes, and people to overcome virtual stall will end up driving advances in virtualization – not just increasing the number of VMs deployed, but improving their virtualization maturity. This in turn will drive not just the incremental (albeit short-lived) CapEx savings from server consolidation, but also to fundamental and long-term gains in OpEx reduction, business agility, service availability, continuity, and more that comes when virtualization (and cloud) transforms from a tactical IT project to a strategic business enabler.

Of course, I have an opinion (anyone who knows me will not be at all surprised by that!), but I am also very interested in what other people think is going to be the key to Pete’s questions. Please do go ahead and check out the video on CIOZone.com, and please add to the discussion either there or here. Or, like me, both!

Obligatory picture of random cloud

Without pointing any fingers, there seems to be a persistent refrain from some public cloud computing proponents that says, ‘If you are running your own IT, then you are doing it wrong’. This attitude fails to account for the magnitude and value of many legacy investments in people, process, and technology.  It ignores the many challenges and risks posed by migrating enterprise IT to public cloud service providers.

I have no doubt that many organizations will continue to run their own IT, even as they also adopt public cloud services – for very good reasons. At the same time, many will migrate their entire IT environment, wholesale, to public cloud services. I do not see this as even slightly contentious.

In fact, my colleague, Gregor Petri (@GregorPetri) argues very well that this should not even be a debate. I cannot disagree – and in an ideal world I would not be talking about it either – but there needs to be a realistic balance to the ‘private IT is doing it wrong’ crowd.

After all, try telling most Fortune 1000 CIOs that they should tip their entire multi-data center IT investment, along with their entire end-user IT investment, into the dump and put it all on Amazon (like Netflix did). I bet they will laugh in your face before they kick you out of their office and get back to doing real work.  They have a sunk cost in existing IT investments – not to mention a valuable investment in people who know the environment and the business they are supporting. This is also supported by a vast array of IT and business processes that actually make the business more efficient and effective.

Public cloud might be logical for most smaller businesses, new businesses, or new applications like Netflix’ streaming video service, but for large enterprises, completely abandoning many millions of dollars of paid-for equipment, and an immeasurable amount of process and skill investment, is frequently unjustifiable. As much as they might want to get rid of internal IT, for large enterprises especially, it simply will not make sense – financially, or to the business.

To start with, there is the massive cost of rewriting all the existing applications from mainframe, UNIX, i5/OS, Unisys, and NonStop (at least) to run on commodity servers and infrastructures. Of course, this is not just a porting exercise. Migrating to public cloud would require completely new architectures for most enterprise applications, not least to accommodate the higher impact (though not necessarily greater frequency) of the downtime endemic in cloud computing. Moreover, given the typically high utilization, performance, and throughput of most of these ‘legacy’ platforms, the cost benefit of migration to commodity systems is questionable at best.

Deploying all new applications to the cloud and just letting legacy applications die through attrition will not allow wholesale migration to public cloud providers either. Some enterprises have applications that are 10, 20, or even 30 years old and still running critical workloads. Today’s crucial applications are going to be around – and running on private ‘legacy’ systems – for a long, long time.

In many cases, regardless of financial factors, it is not even desirable to move enterprise IT into the cloud. Despite all the failings – and they are typically legion – of existing investments in ‘legacy’ people, process, and technology, they frequently do still deliver substantial value to the business. Moreover, they have baked-in an irreplaceably deep level of experience, commitment, and understanding of the core business. They don’t treat all workloads as equal, and do prioritize the most important services in their portfolios.

Commodity cloud providers on the other hand treat all workloads – and all customers – in a shared environment as commodities. They do not provide the special treatment that some workloads really do require – such as some compliance-bound, high-revenue, or non-stop workloads. They do not provide time or event-based reactions to changing business priorities. They do not make sure to allocate the first servers that come up after a system-wide outage to high-priority workloads based on business policy. I talk to CIOs all the time who are laser-focused on aligning IT to the business; if a public cloud provider doesn’t even understand their business priorities, let alone prioritize them, then that goal is difficult, if not impossible, to reach.

This commodity attitude can be even more deleterious when a business (or their website) comes under attack – such as directly in a DDOS or similar attack, in a public relations campaign, or by a sovereign government – for supporting a controversial cause, providing some information, or otherwise becoming unpopular with some group – private or public. When that happens there is no guarantee that your cloud service provider will not just pull the plug on ‘your’ IT service for fear of public backlash, or under pressure from some government force, covert or otherwise (as recently happened with Wikileaks and with Florida pastor Terry Jones). They may even just ‘bump’ one business workload for another, simply because they have oversold capacity – a common practice in all sorts of shared service industries (including transportation, telecommunications, utilities, banking, and Web hosting).

For cloud providers, each supported IT service is just part of their revenue. If hosting any given IT service is not profitable to them (and/or causes them a public relations or legal problem), whether they cancel your contract is simply an ROI calculation. They are not going to fight their customers’ legal battles; they are not going to stand up to a autocratic (or even democratic) government; they are not going to risk their whole business for the sake of one customer. If the service provider does pull the plug, the business is likely to be left not only without an IT infrastructure, but possibly without an offsite backup to restore continuity on another provider – assuming another host will even take them on.

For many enterprises then, moving their private IT to public cloud service providers would not just add cost, but also add additional management burdens, compliance issues, security threats, and business risks. For enterprises that can operate within their own scalable and dynamic data center, public cloud is not ‘your mess for less’; it is ‘more mess for more’.

Moreover, in some cases there may not even be available (or possible?) cloud computing solutions – for example, in low (or no) bandwidth environments, delivering POS or ATM infrastructure, hardware-dependent back-office systems, or high-volume distributed end user computing.

However, there is no reason why private IT cannot gain at least some, if not all, of the benefits of public cloud. Most large enterprises can, will, and should use virtualization, automation, and service management to build elastic resource pools, allocate them to fixed and variable service requirements, and effectively deliver on-demand computing.

With continuous capacity management, integrated with performance monitoring, provisioning, and configuration management, this is possible even without over-investing in spare capacity. Even though they don’t actually have ‘unlimited’ compute resources, large well-managed private IT systems can appear infinitely scalable, at least as much as cloud service providers can. It would actually be interesting to compare the IT resources of major global enterprises with Amazon and other service providers. I would bet that enterprises like Wal-Mart, Citigroup, General Electric, etc. (not to mention many governments) actually have more compute resources to allocate in a dynamic cloud than most of the supposedly ‘infinite’ cloud providers.

With highly automated IT solutions, enterprises like Qualcomm have already delivered these benefits with their own private internal cloud. Others have been so successful they are now public cloud providers themselves – like Telstra Australia and Verizon. In fact, Gartner is actually predicting that by 2015, 20 percent of non-IT Global 500 companies will be cloud service providers. Moreover, there is even evidence that over time, investment in private IT infrastructure is actually more cost-effective than outsourcing it to external cloud providers, especially for larger enterprises.

I am certainly not saying enterprises should avoid public cloud computing entirely. That is just as absurd as the converse. Public cloud computing provides incredible opportunity, especially for small and mid-sized businesses, but also for enterprises. Without doubt, many workloads absolutely should be relocated to public cloud providers; some businesses probably are doing it wrong by running any IT of their own.

And perhaps in 20 years all these problems will  be solved. But I do not think it is very useful to rely too much on what might happen ‘in the long run’. It is an interesting exercise, and certainly can help inform long-term strategy, but as Keynes famously said, “The long run is a misleading guide to current affairs. In the long run we are all dead.” Especially predicting some hypothetical future state where all problems are solved is just a bit too convenient.

However, framing cloud computing as ‘all or nothing’ is a false dichotomy, because there is a realistic and hugely popular third option: a hybrid cloud. Indeed, most real-life CIOs are actually planning or deploying this model today, where both internal and external IT are combined in the best possible ways to drive business value. Most enterprise CIOs running their own IT are also engaging in both evolutionary and revolutionary approaches to cloud; managing a hybrid supply chain of public, private, and hybrid IT; and delivering a complex mix of IT services.

They are not ‘doing it wrong’ – they are doing what they need to do, with what they have, to make their businesses successful.

I was pointed the other day to a chart on the Business Insider ‘Chart of the Day’ (@chartoftheday) showing the R&D expenditures for a handful of tech companies, evidence of Apple’s supposedly superior ‘innovation’ compared to four apparently randomly chosen tech companies.

On the surface, I thought it was an interesting idea, so I looked at R&D spending in companies that are actually related, in the virtualization and cloud computing space. With a little research on Google Finance, I put together the following chart:

Source: Google Finance/SEC

While it is interesting to look at these numbers, and individual comparisons can be somewhat revealing, I don’t see a reliable correlation between technology innovation and R&D spending – either as a percentage of revenues, or an absolute amount.

I’ll just leave it here though. Feel free to comment on what you think this means.

Appendix: For the Inquisitive

If you are geek like me, you are probably wondering exactly what R&D means in context of publicly filed SEC documents. For the Generally Accepted Accounting Principles (GAAP), R&D is defined as follows

a. Research is planned search or critical investigation aimed at discovery of new knowledge with the hope that such knowledge will be useful in developing a new product or service (hereinafter “product”) or a new process or technique (hereinafter “process”) or in bringing about a significant improvement to an existing product or process.

b. Development is the translation of research findings or other knowledge into a plan or design for a new product or process or for a significant improvement to an existing product or process whether intended for sale or use. It includes the conceptual formulation, design, and testing of product alternatives, construction of prototypes, and operation of pilot plants. It does not include routine or periodic alterations to existing products, production lines, manufacturing processes, and other on-going operations even though those alterations may represent improvements and it does not include market research or market testing activities.

Financial Accounting Standards Board, Statement of Financial Accounting Standards No. 2–Accounting for Research and Development Costs, October 1974, p5

The source also has a number of useful examples of what is and is not, considered R&D. Of particular note, the following are not included in R&D costs:

• On-going efforts to refine, enrich, or otherwise improve upon the qualities of an existing product
• Adaptation of an existing capability to a particular requirement or customer’s need
• Legal work in connection with patent applications or litigation
• Intangibles that are purchased from others … that have alternative future uses

You may also want to check the data for yourself too. It all comes from the most recent annual reports, as recorded in Google Finance. I have included the data below, with links to the original Google Finance pages. Hopefully I have transcribed it correctly. Enjoy!

Federal CIO Vivek Kundra and the CIO Council

If there was still any doubt about the real world use cases for cloud computing, the US Federal Government last week published a 38-page report  entitled “State of Public Sector Cloud Computing” (link to PDF at CIO.gov). Attributed to the Federal CIO Vivek Kundra, it is stamped with the seal/logo of the CIO Council, which comprises the CIOs of some 28 federal government agencies.

The report details 30 case studies in public sector cloud computing (for both state and federal governments), covering IaaS, PaaS, and SaaS service models; using private, public, community, and hybrid cloud deployment models; with both on-premise and off-premise implementations.

Measurable Benefits from Key Case Studies

After perfunctorily reciting what it calls “the broadly recognized and adopted NIST Definition of Cloud Computing,” and using the opportunity to briefly push its own barrow on cloud standards (a subject I plan to blog about in more detail at another time), the report cites several projects with ‘soft’ outcomes – improved productivity, better efficiency, higher reliability – as well as several planned cloud projects that are yet to bear fruit.

However, most of the report is given over to demonstrating solid and measurable outcomes from over a dozen current cloud deployment case studies involving multiple state and federal government agencies, with cloud success stories such as:

• The US Army is piloting a customized version of Salesforce.com to update its 10 year old recruiting systems for Web 2.0, social media, mobile devices, marketing integration, real-time data interchange, and engagement tracking. At an annual cost of $54,000, this pilot compares to bids from traditional IT vendors ranging from $500K to over $1 million, and has already replaced five traditional recruiting centers.
• The Department of Health and Human Services is also using Salesforce.com to support the implementation of Electronic Health Records systems. This new CRM system for working with participating healthcare providers was deployed in just 3 months, instead of the full year estimated for an internally delivered system.
• The General Services Administration (GSA) moved to a Terremark Enterprise Cloud service, to take advantage of on-demand scalability for Web sites like USA.gov. As a result, GSA accelerated its site upgrade time from nine months to a maximum of one day, reduced monthly downtime from roughly two hours to near zero (99.9% availability), and reduced annual costs for USA.gov by $1.7 million, from $2.35 million to $650,000, or 72%.
• The Defense Information Systems Agency (DISA) is using virtualization with a self-service portal to provide on-demand server space for development teams. With just an approved Government credit card, these end users can set up new environments (with DoD-compliant security guaranteed) in just 24 hours – down from three to six weeks – and at a “reasonable” cost.
“DISA estimates PaaS cloud savings between $200,000 and $500,000 per project.”
• DISA also used cloud provider CollabNet to set up Forge.mil, a private PaaS cloud development environment with a heavy focus on collaboration and code sharing/reuse. DISA estimates this saves between $200,000 and $500,000 per project – not including the estimated $15 million in cost avoidance by utilizing an open source philosophy.
• The Lawrence Berkeley National Labs (LBL), part of the Dept of Energy, is using Google Apps for 2,300 e-mail users, and planning to more than double that by August. LBL estimates they will save $1.5 million over five years “in hardware, software and labor costs from the deployments they have already made.”
• NASA’s Jet Propulsion Laboratory used a Microsoft Azure development platform “to excite the public about Mars” with the website, BeAMartian.jpl.nasa.gov. This site has generated over 2,000 pieces of social media, inspired 200 traditional media stories, responded up 2.5 million API queries, gathered  40,000 votes in its ‘Town Hall’ polls, and attracted 5,000 registrations from individuals and teams.
• The Federal Labor Relations Authority recently replaced its underperforming, decade-old case management system, switching to Intuit’s Quickbase system. As a result, it was able to go from requirements-definition to completed development in 10 months – a quarter of the original deployment time – and expects a TCO reduction of nearly $600,000 over five years.
“Moving Recovery.gov to Amazon EC2 will drive cost savings of $750,000”
• Less than a month ago, the Recovery Accountability and Transparency Board moved Recovery.gov to a “fully scalable site” in the Amazon EC2 infrastructure cloud, delivering “added security” and “nearly 100 percent uptime.” The Board is projecting that this move will drive cost savings of $750,000 through FY2011 (4% of its $18 million budget) – while allowing it to reallocate more than $1 million worth of hardware and software.
• The New Jersey Transit Authority also used Salesforce.com (alongside some organizational change) to improve its customer service system. The new cloud-based processes allowed the same number of staff to handle 5 times the number of enquires (from 8354 in 2004 to 42,323 in 2006), reduced response time for enquiries by 35%, and improved productivity by 31%.
• Wisconsin’s Department of Natural Resources replaced its aging video conferencing systems with Microsoft LiveMeeting as an alternative to server-based collaboration software. Since migration in 2009, this has saved an estimated $320,000, with ROI expected to grow from 270% for the first year to over 400% in future years.
• The State of Utah uses several public cloud services (Force.com, Google Earth Pro, and Wikispaces), and has completed 70% of its private cloud project to move 1,800 physical servers in over 35 locations to a virtual platform of just 400 servers. The private cloud project alone is expected to the state save $4 million annually – over 2.5% of its $150m IT budget.
• Facing a $400 million deficit, the City of Los Angeles has been transitioning to Google Apps cloud-based e-mail, with all employees to be cut over by June 30 this year. The City’s CTO estimates a direct savings of $5.5 million over 5 years, and a total ROI (including increased productivity) of $20-30m.
  “Colorado estimates annual savings of $8m, and up to $20m in expense avoidance”
• The City of Orlando rolled out a similar Google Mail project for all 3,000 city employees in January this year. The City has realized a 65% reduction in e-mail costs, not including benefits from improved productivity, increased storage allocation (from 100MB to 25GB per user), improved security/malware detection, and enhanced mobile device support.
• The State of Colorado is shifting to a hybrid cloud model, mixing private cloud (an existing data center leveraging server virtualization), a virtual private cloud (for additional pay-as-you-go scalability), and public cloud (Google Apps for e-mail and office productivity). Just by shifting 122 servers running Lotus Notes, Microsoft Exchange, and Novell GroupWise to the cloud, Colorado estimates annual savings of $8 million, and up to $20 million in expense avoidance over 3 years.

Set SMART Goals, But Be Pragmatic

Kundra does not shy away from clearly stating his ongoing cloud computing goals in this report. By 2011, all business cases for new federal IT investment must include cloud alternatives; by 2012, all enhancements to existing systems must do the same; by 2013, all IT investments, even on legacy systems, must be justified against a cloud alternative. These SMART (Specific, Measurable, Attainable, Relevant, and Timed) goals are important to overcome the all-too-frequent adoption of disruptive technologies almost as a fad, unrelated to business goals and without a clear and realistic timeline.

However, these case studies show an essential pragmatism about the public sector approach to cloud computing. Kundra and the CIO Council recognize (as I have previously published) that the cloud will not completely replace on-premise IT, stipulating:

“Federal agencies are to deploy cloud computing solutions to improve the delivery of IT services, where the cloud computing solution has demonstrable benefits versus the status quo.”

So while cloud must be increasingly evaluated, actual cloud adoption must be justified by “demonstrable benefits” that improve IT service delivery, not just reduce costs. As I have stated in EMA research and blogged about here, it is important for enterprises (public or private) to “look for opportunities, and do what makes sense” when it comes to cloud computing. This is reflected by thought-leaders like Gartner’s Thomas Bittman (@tombitt), who explains that for some organizations “a 70% private cloud is absolutely good enough.”

Cloud Lessons For Other CIOs?

These case studies have a lot of lessons to offer other business and IT leaders, both private and public sector, in everything from mid-sized businesses to the largest enterprises. They detail many clear and realistic case studies; provide insight into achieving both specific ROI and soft benefits; show how cloud can be applied to both business- and IT-oriented goals; and give ideas for how CIOs might address real problems with cloud alternatives.

Moreover, more than any set of self-published corporate case studies, this is incredibly significant, because, as the report points out:

“The United States Government is the world’s largest consumer of information technology, spending over $76 billion annually on more than 10,000 different systems.”

This level of influence from the world’s largest consumer of IT will drive a solid and relentless march to cloud computing, a juggernaut that will likely carry the rest of us along, whether we like it or not.

However, it reads almost like promotional material from a cloud provider – which, in a way, it is – because it does not deal directly with any of the potential problems of cloud computing. It mentions security only very briefly, and then only how certain cloud implementations actually improve security (with no details). It does not give any details of how federal clouds have ensured compliance with regulations like the Federal Rules of Disclosure and DOD 5015, and industry requirements like PCI-DSS. It does not talk about if, or how, they overcame the endemic  problems of performance assurance and continuity in the cloud. Perhaps most ironically of all, it does not even mention how it overcame the tough political and departmental challenges that are cited by analysts as one of the top barriers to both virtualization and cloud adoption.

So for CIOs, this report really needs to be taken with a grain of salt. Be informed and educated by these case studies; use them to be set pragmatic expectations and SMART goals; but be wary that as much as it says about the upside of cloud computing, it avoids saying just as much – if not more – about the potential for deleterious, or even disastrous, downsides.

There is a perennial debate in cloud computing about whether a failure of one cloud service provider can be more generalized to a ‘failure of cloud computing’. It is an important question because availability is a key decision factor in choosing between private and public cloud, and between public cloud providers.

The most recent example of such failures is the power outage at IaaS provider Rackspace’s London facility, but of course, we have seen this before from many public cloud providers – including Rackspace in particular, and not just once. SaaS provider Salesforce.com (and its PaaS arm, Force.com) has also had one outage already this year, an event that is far from unusual, and nothing new. Amazon, Yahoo, Microsoft, GoGrid, RIM, Twitter, Paypal and many others have also had substantial (and often repeated) outages.

There are some who dismiss these failures as one-offs, write off partial or short-term failures as too low-impact to matter, or just give poor DR a pass because it is the cloud, and we should not expect any better. Others reach to find semantic differences, calling it a service outage, an application failure, a facilities outage, a power outage, or a resource shortage. Some just redefine cloud to include only those services that did not go down this week (bonus points for adding a vainglorious reference to the ‘real cloud’ or ‘true cloud’).

YMMV, but I don’t see it that way at all. With so many repeated failures in so many cloud providers, these are not just one-off failures. They don’t just happen to isolated providers, they happen across the board. Regardless of the cause – the application, the facilities, the power supply, the lightning rod – an outage of a cloud service provider is still a cloud outage. And the definition of cloud I use is not dogmatic enough to exclude any of the providers that I have cited (and others), let alone define a ‘true cloud’.

So I see every reason to believe that downtime in the public cloud is not the exception, it is the rule; that outages in the public cloud are endemic, and they are systemic.

However, this judgement is absolute, not relative. Failure in one cloud provider may (and I believe does) implicate all cloud providers, but it does not imply downtime is more of a problem in the public cloud than in traditional enterprise IT. Indeed, there is a strong argument that enterprise IT has as many if not more outages, so uptime and availability is no worse in the public cloud than with traditional IT.

In fact, EMA research has shown average enterprise IT uptime is just ‘two nines’, at 99.5%. For a 24×7 system, that is over 50 minutes of downtime, each and every week. Contrast this with public cloud providers. Even with their problems, Amazon EC2 offers a “reasonable effort” to deliver an annual uptime of at least 99.95% – or about 5 minutes downtime per week – and offers a 10% credit for “eligible” breaches. Google guarantees ‘three nines’ (99.9%) uptime for its Premier Edition, or around 10 minutes downtime per week (although it promotes a study that claims an average downtime of 15 minutes a week). The Rackspace SLA promises network, HVAC, and power will be up 100%, though it does not guarantee server availability (beyond promising a 60 minute maximum repair window), and all promises exclude ‘scheduled maintenance’.

So for the average enterprise, ‘normal’ cloud computing outages, while endemic, can still be 5 to 10 times less frequent than in their own data centers.

However, it is not a black and white issue, not least because a focus on broad uptime percentages or on single instance failures ignores the huge nuance behind a single uptime number.

For example, many environments report ‘five nines’ (99.999%) or even 100% uptime – less than one second of unplanned downtime each day – for their critical systems by using processes and tools for high availability, fault tolerance, asset maintenance, live migration, etc. EMA has also found that best performers in Virtual Systems Management – 15% of enterprises – report an average of five nines uptime.

If they need to, enterprise CIOs can invest in technology to provide two, three, four or five nines uptime within their own data center. They can implement redundant hardware, HA and FT, multi-site replication, and more – if they want to pay for it. They can monitor for outages, know exactly when they happen, and react automatically to fix them immediately (or even use predictive analytics and automation tools to avoid them entirely). They can provide this as required, as a value-add to their business unit customers, or as an additional charge (or at least an exposed cost)  to the business to let them choose how critical their applications really are.

However, with the public cloud, neither the business nor the CIO has any real choice. With few or no management or automation tools, public cloud providers simply do not currently offer the same flexibility and accountability as internal IT. Without good management tools, no public cloud provider currently matches enterprise IT at the higher mission-critical reaches of availability.

So, this fight does not end in a knock-out for either side. As is common in the real world, nothing is black and white, but rather many shades of grey.

In the end, the solid achievements of public cloud providers, despite the bad press, does not absolve them of any blame or negate generalizations of downtime being endemic in the public cloud. However, the relatively poor performance of enterprise IT on average still does not ensure public cloud will be any better in any specific cases.

What this does show, however, is that CIOs who are planning to build their own private cloud have a surprisingly high bar to reach. They should not dismiss public cloud options out of hand, but rather should strongly consider whether they can realistically and cost-effectively meet the three, four, and even five nines that public cloud providers guarantee.

I am getting so sick of the continual bickering over definitions of cloud computing. Even more frustrating is the hype from all the vested interests – vendors and analysts, mostly – trying to define cloud computing in ways that they imagine will best contribute to their own commercial success. And I know that I am not alone.

What is wrong with the definition that the US National Institute of Standards and Technology (NIST) – a division of the US Department of Commerce – uses?

You can read the entire definition online [link updated 8/12/11]. It is only 2 pages. Here, for the unaware, is the meat of it:

“Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”

Does this suck so badly that every [insert your preferred expletive epithet here] needs a new definition?

It goes on to include:

• Five essential characteristics: On-demand self-service; Broad network access; Resource pooling; Rapid elasticity; and Measured Service.

• Three service models: Software as a Service (SaaS); Platform as a Service (PaaS); and Infrastructure as a Service (IaaS).

• Four deployment models: Private cloud; Community cloud; Public cloud; and Hybrid cloud.

So what exactly is wrong with that?! Why does every man and his dog feel the need to throw their own definition of could computing into the ring?

Don’t get me wrong. Definitions are important. Definitions enable a common understanding of terminology, essential when talking about complex technologies. And I have pushed my own definitions before (like my definition for virtualization, widely adopted after Wikipedia picked it up in 2006).

But why fight city hall (in this case, almost literally)? NIST has a very elegant definition that is:

• Intelligent – it has been through (to date) 15 iterations, and has accepted input from many of the brightest minds in cloud computing (while presumably ignoring some dimmer bulbs)
• Independent – it is from a mature, well-established, and exceptionally talented US government agency, which is both apolitical, and science-based
• Commercially agnostic – it does not specify that anyone needs to be making money, nor does it preclude it, allowing cloud to be B2B, B2C, B2G, G2C, or any other model
• Accommodating – all established cloud vendors (like Amazon, Google, Rackspace, Salesforce, and others) fit into this definition, as well as private and government models.
• Clear – it is not full of jargon or ‘cloudwash’, but rather has easily understood, plain English concepts that are not only unambiguous but also usefully prescriptive
• Comprehensive – it includes all the important core concepts such as self-service, resource pooling, rapid elasticity, accessibility, usage costing, multiple use cases, and more
• SMART – it does not try to create anything exceptional or outrageous, but does define a set of Specific, Measurable, Achievable, Relevant, and Timely objectives

We trust NIST to define the official time for all of the United States. We trust it to calibrate instruments for NASA. We trust it to supply “industry, academia, government, and other users with over 1100 reference materials”.

Moreover, this is what the US government is using to define cloud computing, as noted by Vivek Kundra (the US Federal CIO). Indeed, Kundra has strongly indicated that the US government will be one of the strongest, largest, and most important proponents, providers, and consumers of cloud computing (cf. sites like apps.gov and data. gov). Other levels of government – and even other nations – will almost certainly follow their lead, and the NIST definition of cloud computing.

So why can’t people trust NIST with the definition of cloud computing, and just get on with the job of solving real problems for their customers? Bickering and chest-beating over self-enriching definitions is not needed, it is not useful, and it is not helpful.