The 'new normal' makes cloud mandatory, not optional.

Today I was published in one of the top cloud computing journals. In fact, it is the Cloud Computing Journal, part of the SYS-CON stable and the same organization that runs the excellent Cloud Expo events. The article is called “Eleven Tips for Successful Cloud Computing Adoption“:

Key issues can make or break an organization’s strategic cloud adoption. The intersection of cloud computing with business strategy, Big Data, vendor lock-in, globalization, collaboration, security, licensing, virtualization, confidence, and the ‘new normal’ can act as huge points of concern. So I put down some thoughts on this, and ended up – in no particular order – with the following 11 tips for the successful adoption of cloud computing:

Please read the whole article at the Cloud Computing Journal.

—

So what do you reckon? Are these tips useful for you? What tips did I miss? I would love to see your comments at Cloud Computing Jounal, in my comments section below, or as always on Twitter.

So, here is something interesting I discovered today, courtesy of a tweet from Christian Reilly (@ReillyUSA) – the US federal agency, the National Institute of Standards and Technology (NIST), today released Version 1 of their Cloud Computing Reference Architecture (PDF). It is free and, like all US Federal Government content, it is open.

I have written about NIST before – both in my research work at EMA and in my personal blog – and wholeheartedly endorse their excellent definitions for cloud computing. If we can trust them to define time – and a thousand more standards besides – we can trust them to define cloud.

So I am more than willing to let them have a go at describing a cloud reference architecture.

The document essentially provides a brief outline of the five key actors:

• Cloud Consumer – Person or organization that maintains a business relationship with, and uses service from, Cloud Providers.
• Cloud Provider – Person, organization or entity responsible for making a service available to Cloud Consumers.
• Cloud Auditor – A party that can conduct independent assessment of cloud services, information system operations, performance and security of the cloud implementation.
• Cloud Broker – An entity manages the use, performance and delivery of cloud services, and negotiates relationships between Cloud Providers and Cloud Consumers.
• Cloud Carrier – The intermediary that provides connectivity and transport of cloud services

Then through a combination of definition, example, and illustration, it places these actors into a big picture end state ‘reference architecture’:

NIST Cloud Reference Architecture V1

Despite some clear flaws, I think this is a great document. More than just a series of definitions, far less than a ‘true’ technical reference architecture, it is advisory and high-level, but practical and usable.

Some key standouts for me include:

‘Grown-up’ management finally takes center stage

I am particularly excited that such a powerful voice in cloud computing is finally highlighting the primary importance of management in their cloud documentation. Almost half this document is focused in cloud management – something I have been deeply committed to for many years. It does not just rehash simplistic notions of cloud – that it is just live migration, capacity management, or an orchestration engine. It shows that you need to maintain many mature enterprise management disciplines – even as ‘old school’ as performance management and SLM – as you grow your cloud maturity. All actors – including consumers and providers – must mature as well. You can call it names like ‘legacy’, or pretend ‘enterprise’ is code for ‘mainframe’ – like that’s a bad thing – but NIST clearly believes a cloud computing environment needs mature management discipline.

It’s all about the service

Of the eleven management slides, six are devoted specifically to a concept NIST calls Cloud Service Management (CSM) – something I first wrote about in 2008, and which likely has been around for longer than that. NIST defines CSM as:

all the service-related functions that are necessary for the management and operations of those services required by or proposed to cloud consumers.

It breaks these down into three main management areas as follows:

NIST Architecture for Cloud Service Management

This is a huge step forward in pragmatic (dare I say, responsible?) cloud service delivery. Many vendors are trying to define cloud as advanced virtualization, or rapid provisioning, or service catalog, or automation – or a proprietary ‘cloud in a box’. Others claim public cloud vendors will do it all, as though there is no need to deal with performance assurance, incident reporting, or bandwidth management. A rational, independent, authoritative body explaining the breadth of integrated enterprise service management required to deliver a high quality cloud service is important information for many CIOs who have been led to believe a more simplistic vision.

Service orchestration needs breadth and deep

This cloud reference architecture devotes special attention to service orchestration across multiple layers of the cloud environment:

• physical resources – including hardware  (memory, storage networking, etc.) and facilities (HVAC, power, comms, etc.)
• virtual systems – hypervisors, virtual machines, virtual data storage, and VM platform tools
• physical systems – NIST specifically accommodates non-virtual resources for cloud delivery
• application delivery – top-down delivery of end-user software clients or other programs
• platform delivery – various development environments, databases, app servers, etc
• infrastructure services – processing, storage, networks, and other fundamental resources

In clearly delineating the need for sophisticated process automation, real-time management, and integration, it shows the need to orchestrate not just a single platform or silo, but end to end across multiple layers, platforms, technologies, and vendors.

The value of an independent judge

I also really like the idea of specifically including an independent arbiter – the Cloud Auditor – that is empowered to “evaluate the services provided by a Cloud Provider in terms of security controls, privacy impact, performance, etc.” The need for an independent reviewer is already well overdue. Today, even cloud leaders like Amazon, WordPress, Salesforce, and Netflix can be down for hours with no reporting or explanation, and with no more payback than a sorry letter and few pennies in credit for time lost. They are also killing off any expectation of security, compliance, or privacy by hiding away fine print like the right to “access, retain, use and disclose your account information and your files … as [they] determine is necessary”. In this climate, we already desperately need an independent agent to adjudge the operations, performance and security of all cloud providers, especially public cloud providers.

However, the reference architecture is not all good, and some significant issues also stood out for me:

Security is a one-sided activity

The reference architecture hangs the responsibility for security almost entirely on the Cloud Provider, which is poor advice. Unlike cookies, security is not a ‘sometime’ food, and active participation in security cannot be attributed to any one actor or interaction. For example, two-factor authentication necessarily requires active participation by both service provider and service consumer. Cloud Auditors and Cloud Brokers also have significant responsibilities for security.

Everyone is a ‘Cloud Carrier’

The ‘Cloud Carrier’ actor essentially elevates all telcos to the role of ‘Cloud Carrier’ with no change in business model or technology. It also actually classifies cabs, couriers, and even the UPS as ‘Cloud Carriers’, as this actor includes any provider of “physical transport of storage media such as high-capacity hard drives.” A requirement for Cloud Providers to set SLAs with Cloud Carriers is especially unlikely for a public cloud, though it makes more sense in the context of a private cloud. It also leads to difficult questions of carrier interoperability, quality of service, traffic shaping, and even ‘Net neutrality.

Encryption is optional

Encryption is included as an optional (!) activity, which of itself is unacceptable for mission-critical enterprise applications. Even then it is ascribed to the Cloud Carrier. The idea sounds great – carriers provide “dedicated and encrypted connections” for the “connectivity and transport of cloud services.” However, it is unrealistic for carriers to implement interoperable encryption for ‘cloud traffic’ (whatever that is). It also forgoes the current, quite logical, de facto standard – encryption directly between the provider and the consumer, regardless of carrier. Again, much of this architecture seems to be more directed at private cloud networks, rather than public networks including the Internet.

Privacy is not having to say you’re sorry

Privacy is included as a single line item without much meat on the bone:

Protect the assured, proper, and consistent collection, processing, communication, use and disposition of personal and personally identifiable information (PII) information on the cloud.

This is so neutral as to be unhelpful. Sharing personal data with advertisers, handing over corporate data to warrantless investigations, or even selling your customer database on eBay, may all be ‘assured, proper, and consistent’ according to some so-called ‘privacy policies’. The document does allow the Cloud Auditor to “evaluate the services provided by a cloud provider in terms of … privacy impact,” but beyond this it has no advice on what privacy actually means. Perhaps this is asking too much of a high-level document, but personal privacy and data loss prevention are critical issues in cloud computing. From controversy over Facebook’s exposure of personal details to cloud providers cutting off legitimate businesses, there is significant concern over privacy. I expected more prescriptive advice, rather than a neutral academic definition, especially from a public body setting policy for the IRS, the Pentagon, Department of Social Security, and other sensitive departments.

Management is entirely a provider activity

NIST attributes cloud management – including security and service management – entirely to the cloud provider. This is rare (if it exists at all) among public cloud providers today, and is unlikely to ever be acceptable for most enterprises. With providers doing all the management the fox is watching the hen house. Consumers will require at least some participation. We learned that it is bad to give total control to third-party providers when we did things like IT outsourcing. Just as with cloud computing itself, the majority of enterprises will probably always want a hybrid model for cloud management.

The bottom line

I do see this as a very useful document. It is really quite good – as far as it goes. It is also a very important document – for what it gets right, for what it gets wrong, and for where it comes from, as NIST is helping to shape cloud standards for the world’s largest consumer of information technology. It is far from perfect, and I believe has some truly fundamental flaws, but it is only a Version 1, and who among us has delivered a perfect product on the first release?

So ultimately, it is good enough for now, but I am very much looking forward to the ongoing development of this document. To quote Raymond Umerley (@SecJitsu):

'Close Enough for Government Work'

More than four reasons count towards VM stall

I recently saw a great article in IT World Canada titled “Virtual stall: What it is and why you have it,” written by Jay Litkey, that took up my idea of VM stall, which I first came up with in my blog from May ‘Is “VM Stall” the Next Big Virtualization Challenge?‘.

Though they barely acknowledge my blog as their inspiration (and as a competitor to CA Technologies – my employer – why would they?), it seems Jay and his team have wholeheartedly taken up my concern with VM stall, and not just in the IT World Canada article. Marketing lead David Lynch was quoted on the topic in a post by Bruce Hoard of Virtualization Review, and in a recent Tech Target article on ‘ISV stall’. Several posts on their corporate blog also address the issue as if it was their own baby.

In my past life at EMA, I have spoken with both Jay and David a number of times, and had a lot of time for what they were doing in the management space. For a small startup with limited resources, it is great that they can take the time to pick up my idea and run with it.

The IT World Canada article is really worthwhile, because it zeroes in on some important concepts. It helps to expand the thought around VM stall, and specifically on a couple of additional causes, as it notes:

Virtual stall has four main causes:

• Scalability issues:  A single IT team often finds it difficult to scale beyond the 25-30 per cent penetration range. This is due to the combination of lack of automation and reporting in virtualization management tools, creating time-consuming manual processes that are a particular problem when there is a lack of experienced and trained staff.
• Management issues: The data centre is not a place that can be managed manually; there are too many elements to be checked, and too many independencies [sic]. And, while there are levels of automation built into the virtualization platform, they can be difficult to define and implement. The lack of automated monitoring, alerting and control becomes more and more of a problem as the overall level of virtualization in the data centre increases.
• Process issues:  Enterprise virtualization impacts a wide range of existing data centre processes, all of which need to be modified, replaced, or augmented. As long as the virtual environments are small and self-contained, these processes can be manipulated or ignored. But as the environment grows, it reaches a point when they have to be dealt with before real efficiencies can be reached. The more “process-mature” an organization is, the more quickly this point is reached.
• Co-ordination issues: Virtualization crosses multiple silos and ultimately requires a level of co-operation and integration that is impossible to achieve with the traditional silo management structure. In addition, the first workloads to be virtualized tend to be less critical ones.  However, as environments grow, higher-risk, higher-impact services are virtualized. These tend to have more stakeholders, more politics, more distributed infrastructures, and a greater cost of failure and downtime. Consequently, they require more coordination.

This is great insight, and offers a number of important causes. However, I don’t think it is reasonable to say there are just “four main causes.” Not to pick on Jay, as it is probably just unfortunate phrasing, but I think it is important to see that the issues of VM stall are much more varied, complex, and numerous.

I am not entirely without fault either. To start with, when I first identified the issue of VM stall in my blog post back in May, I said that “I see many possible causes for VM stall,” but like Jay I only identified four examples. As Jay recounts in his analysis, I saw scalability and manageability as key issues; but unlike Jay, I chose to highlight risk aversion and resourcing as two more of my examples.

However, even these six are just a part of the problem. As I said when I spoke with my great mate (and one of the industry’s great virtualization gurus, observers, and commentators), David Marshall of Hyper9 and InfoWorld in his article, “VM stall: Breaking through the second phase of virtualization“:

“… many organizations strike a ‘perfect storm’ of challenges that slows their virtualization rollout, or stops it entirely. Some causes at this stage include greater complexity of services and applications, higher demand on scarce virtualization skills, limited visibility into a growing deployment, increasingly heterogeneous systems, and greater resistance from risk-averse application owners and recalcitrant application vendors.”

In the same article, David spoke with Dave Bartoletti, formerly of automation vendor Enigmatec and now a leading light showing the way through the virtualization darkness with research and advisory analyst firm, the Taneja Group:

“The second wave of issues is always harder when a core technology matures. Server virtualization essentially paid for itself in CAPEX savings, but when we virtualize Tier 1 business-critical applications, or user desktops, CAPEX savings take a backseat to application performance and IT efficiency, and this is why we’re stalling.”

My former editor at Tech Target and another keen virtualization observer, Colin Steele, highlighted another core element of VM stall, in his article “ISV stall makes virtualizing applications a challenge“:

By now, the benefits of virtualizing applications are clear, but the goal of 100% virtualization remains elusive. One reason is that some independent software vendors (ISVs) don’t support their server-based applications — databases, telecom apps, healthcare programs, etc. — on virtual servers.

Moreover, I talk a lot with customers about their real world concerns, so I can quickly pinpoint many other causes. They talk to me about issues like vendor licensing, facilities constraints, capacity blindness, service prioritization, deployment costs, line-of-business resistance, internal politics, a lack of skills, and even senior management resistance.

In fact, last week at CA Expo in Australia, I talked with CA Technologies customers about seven significant issues in virtualization that are contributing to (among other things) VM stall, as you can see from one of the slides from my presentation:

Virtualization is not clear sailing - from CA Expo Australia

(You can see the whole deck at the CA Expo site)

To be fair to Jay and his team, other posts on his corporate blog agree with me, citing issues like mission-critical apps, management skepticism, bureaucracy, poor project vetting, and more.

I am really glad to see my thoughts around VM stall have captured the imagination of the market. Thanks to Jay for taking this up, and to his team for joining me and CA Technologies in raising awareness of issues causing VM stall.

However, I think we all need to be careful about being categorical about VM stall. It is important to be clear that VM stall – like most enterprise IT issues, and indeed most organizations – is both complex and varied, so trying to categorically define four (or six, or seven, or really any number) of causes for VM stall is underestimating this important problem.

But if we can all contribute new ideas to the community, we will all learn more, and our enterprise customers will benefit from our combined wisdom.

Is old-school physical security really 'good enough' for virtualization?

Whatever happened to virtualization security?

Back in the day, everyone was talking about blue pills and red pills, about sideways attacks and DOM-0 threats, about security profiles and isolation policies, about perimeter defense and security embedded in the hypervisor.

Then, all of a sudden, the buzz seemed to disappear. It really seems like organizations simply don’t have the time, money, desire, or otherwise to pursue dedicated virtualization security.

Indeed, it seems like most of the pure-play virtualization security vendors have folded, been sold, or reworked their strategy.

For example:

• Blue Lane ended up being sold to VMware, reputedly at a bargain price, after failing to get any traction.
• Third Brigade was rolled up into Trend Micro, and now offers a solution for combined ‘physical, virtual and cloud’ protection.
• Reflex and Catbird have repositioned to highlight their value in configuration, compliance, and/or systems management (in addition to their security value).
• Tripwire and Configuresoft have long promoted some virtualization security values, but were never really pure-play virtualization security vendors.
• Even security specialists like Symantec and RSA do not push virtualization security products, preferring mainly to build on existing security paradigms to support virtualization.

Of course VMware still has vShield Zones and the VMsafe API, but of the ISVs it seems that only Altor Networks still plays strongly in the pure-play virtualization security space.

This barely sustaining demand for pure-play virtualization security was reinforced last week in new research from Prism Microsystems (PDF), a software vendor in the SIEM market* (which I learned about in eWeek via @JSchroed). Possible vendor/sample bias aside, this research showed quite starkly how many respondents are securing their virtual environment using traditional (or no) security, and how few are using virtualization-specific security:

Source: Prism Microsystems, ‘2010 State of Virtualization Security Survey’, April 2010

In confirmation of this ennui, Gartner recently predicted at least a 5 year maturity cycle for virtualization security.

All of this is especially perplexing, because there is no doubt virtualization security is still top-of-mind for some very smart and dedicated people. The Prism Microsystems research, for example, says that 86% of its respondents acknowledge that securing virtualization is as important as securing their physical environment.

So I am unclear as to what is causing this lack of market interest. Perhaps CIOs (and/or CISOs):

• saw virtualization security as unnecessary insurance against threats that have never played out ‘in the wild’
• rated the potential financial impact of any additional risks as low enough that they can simply accept them
• believe that vShield Zones and VMsafe are all that is needed (but what about Hyper-V, Xen, etc.?)
• decided instead to invest in management disciplines with more straightforward ROI (virtualization, automation, configuration management, asset management, etc.)
• have simply been unable to justify virtualization security purchases during the economic downturn

Whatever the reason, it really does focus the question: does virtualization security really matter?

In my opinion, it absolutely does. Yet, it seems to me that decision makers are saying that standalone virtualization security is more important theoretically, from a technology and business perspective, than it is in practice. Most enterprise buyers – for better or worse – apparently believe that their existing security paradigns are at least ‘good enough’. They definitely appear instead to be taking classic intrusion detection, data loss prevention, identity & access, and other entrenched security management disciplines, and adapting them to the new technologies of virtualization (and probably cloud as well).

All of which actually does make sense. Without any major virtualization-specific exploits in evidence, perhaps they are right. While it may be valid to take the view that  it is only a matter of time until they are proven wrong, perhaps extending traditional security capabilities into the virtual world is indeed a good approach, at least for now.Perhaps CISOs are actually ahead of the game, integrating management across virtual and physical domains even while their systems and operations counterparts are maintaining virtualization as a silo.

Regardless of whether it is the right approach or not, one thing is apparent – the heat is off the pure-play virtualization security market, at least for now. As CIOs and CISOs focus on applying traditional physical security paradigms on their virtual environments, a different breed of cross-domain, integrated, and extensible tools are proving superior value – at least for now.

A few days ago I was pleased to brief again with Reflex Systems. Apart from the fact that they are doing some very cool things with virtualization management, their approach struck me as, if not unique, at least pleasantly rare.

Good for them!

What I liked most was that they are trying to break down the barriers between systems and security management. Certainly this is something that I discuss regularly with enterprises – the need to stop focusing on silo-based management, and perhaps even more importantly, to stop pandering to silo-focused low-level managers. Almost all of the CIOs, VPs, and IT Directors who I talk with are critically aware of the problems these silos cause – including human errors, resource inefficiencies, security problems, and higher costs.

This is also a constant discussion I have within EMA, especially with the lead of our security practice, Scott Crawford – a brilliant mind on security (amongst many other subjects) who constantly thinks about security in ways I never could or would. We work and publish together on this topic frequently. Indeed, it has come up again in our latest research, which shows that security and risk management are a fundamental requirement for cloud computing – or what EMA calls the ‘Responsible Cloud’.

The upshot of all these conversations is simple – security management and systems management are not, cannot, and should not be completely separate. Not in human terms, not in processes, and not in technologies. Without doubt, anyone in a large enterprise who has ever tried to implement a patch, a configuration change, a firewall update, a software release, or a hundred other data center changes will attest to this in a heartbeat.

Of course (as Scott rightly pointed out when I last spoke with him about this), we will always need security experts, and systems experts – the two disciplines are not the same, and we will always need deep domain expertise in each. So I am not advocating complete convergence. But we need more software tools that provide integration and interoperability that allow these professional to work more effectively together.

While multi-function vendors like CA, Symantec, IBM, and others have the product portfolio to approach these cross-silo problems holistically, there are few ‘best of breed’ vendors thinking this way. Of course, Tripwire and the ever-inspiring Gene Kim (who I have sadly never met) spring to mind for me; so would Configuresoft (although now as part of EMC Ionix, hardly a niche vendor), and the indefatigable Dennis Moreau. Both inspire their teams, technologies, and customers by championing a fundamental understanding that systems and security cannot, at their heart, be completely separated.

(As an aside, these two seem like they would have been incredibly compelling arch-enemies in some ubergeek superhero genre – although I would never want to choose which should be the hero and which the villain!)

I must say that, so far at least, I don’t know the product design team from Reflex personally – guys like Hezi Moore, Aaron Bawcom, and Mike Wronski – as well as I do Dennis or Gene. However, I do know that they all have very credible security chops. Plus, one thing is clear to me.

They get it. They really get it.

And that in itself is a thing of rare beauty.

Deliberately designing functionality that addresses both security and systems management – like functional isolation, integrated access control, change segregation, granular audit trails, policy based management, and role-based access – into a systems management toolset is a rare feat, especially in startup and niche products. It is something I look for all the time, because my enterprise clients often demand it. Sadly, all too often I fail to find it – and I am not even a real security wonk! When I do, I am pleasantly surprised. When I see deep thought going into the security value of a systems management product, I am almost ecstatic.

Unfortunately, the challenge for vendors like Reflex and Tripwire (as it was for Configuresoft, and perhaps is still for EMC, Symantec, etc.) is to find customers that value this synergy. While most high-level IT execs understand this imperative, their holistic view frequently does not translate to many of their lower-level managers, or to many functional IT practitioners.

Of course, there are plenty of departmental ops managers and security managers who do get it. They strive to connect their teams with other groups, driving greater business efficiency and effectiveness as a result. However, unfortunately, many do not, instead focusing on protecting their small empires, walling themselves off from integrated management and cross-functional resourcing.

Similarly, many positive-minded individual technicians will actively seek out cross-skilling opportunities, recognizing that it makes them not just more useful but also more valuable, and more indispensable. However, many practitioners (both security and ops) can be just as bad as the most myopic managers (who they often work for), dogmatically eschewing integrated management tools and processes, seeing them as threats to their own personal domains of control.

Sad but true, best practices like breaking down IT management silos are not always adopted.

Fortunately, vendors like Reflex and Tripwire that have expertise and passion in both ops and security (and – shameless plug – trusted advisors like EMA, which is big enough to have experts in both disciplines, yet small enough that we still work together), are trying to break down these barriers.

And more power to them. They serve their clients much better by promoting the undeniable facts that security values are critical to systems management, and systems management is critical to security.