New Cloud Reference Architecture From NIST

So, here is something interesting I discovered today, courtesy of a tweet from Christian Reilly (@ReillyUSA) – the US federal agency, the National Institute of Standards and Technology (NIST), today released Version 1 of their Cloud Computing Reference Architecture (PDF). It is free and, like all US Federal Government content, it is open.

I have written about NIST before – both in my research work at EMA and in my personal blog – and wholeheartedly endorse their excellent definitions for cloud computing. If we can trust them to define time – and a thousand more standards besides – we can trust them to define cloud.

So I am more than willing to let them have a go at describing a cloud reference architecture.

The document essentially provides a brief outline of the five key actors:

• Cloud Consumer – Person or organization that maintains a business relationship with, and uses service from, Cloud Providers.
• Cloud Provider – Person, organization or entity responsible for making a service available to Cloud Consumers.
• Cloud Auditor – A party that can conduct independent assessment of cloud services, information system operations, performance and security of the cloud implementation.
• Cloud Broker – An entity manages the use, performance and delivery of cloud services, and negotiates relationships between Cloud Providers and Cloud Consumers.
• Cloud Carrier – The intermediary that provides connectivity and transport of cloud services

Then through a combination of definition, example, and illustration, it places these actors into a big picture end state ‘reference architecture’:

NIST Cloud Reference Architecture V1

Despite some clear flaws, I think this is a great document. More than just a series of definitions, far less than a ‘true’ technical reference architecture, it is advisory and high-level, but practical and usable.

Some key standouts for me include:

‘Grown-up’ management finally takes center stage

I am particularly excited that such a powerful voice in cloud computing is finally highlighting the primary importance of management in their cloud documentation. Almost half this document is focused in cloud management – something I have been deeply committed to for many years. It does not just rehash simplistic notions of cloud – that it is just live migration, capacity management, or an orchestration engine. It shows that you need to maintain many mature enterprise management disciplines – even as ‘old school’ as performance management and SLM – as you grow your cloud maturity. All actors – including consumers and providers – must mature as well. You can call it names like ‘legacy’, or pretend ‘enterprise’ is code for ‘mainframe’ – like that’s a bad thing 😉 – but NIST clearly believes a cloud computing environment needs mature management discipline.

It’s all about the service

Of the eleven management slides, six are devoted specifically to a concept NIST calls Cloud Service Management (CSM) – something I first wrote about in 2008, and which likely has been around for longer than that. NIST defines CSM as:

all the service-related functions that are necessary for the management and operations of those services required by or proposed to cloud consumers.

It breaks these down into three main management areas as follows:

NIST Architecture for Cloud Service Management

This is a huge step forward in pragmatic (dare I say, responsible?) cloud service delivery. Many vendors are trying to define cloud as advanced virtualization, or rapid provisioning, or service catalog, or automation – or a proprietary ‘cloud in a box’. Others claim public cloud vendors will do it all, as though there is no need to deal with performance assurance, incident reporting, or bandwidth management. A rational, independent, authoritative body explaining the breadth of integrated enterprise service management required to deliver a high quality cloud service is important information for many CIOs who have been led to believe a more simplistic vision.

Service orchestration needs breadth and deep

This cloud reference architecture devotes special attention to service orchestration across multiple layers of the cloud environment:

• physical resources – including hardware  (memory, storage networking, etc.) and facilities (HVAC, power, comms, etc.)
• virtual systems – hypervisors, virtual machines, virtual data storage, and VM platform tools
• physical systems – NIST specifically accommodates non-virtual resources for cloud delivery
• application delivery – top-down delivery of end-user software clients or other programs
• platform delivery – various development environments, databases, app servers, etc
• infrastructure services – processing, storage, networks, and other fundamental resources

In clearly delineating the need for sophisticated process automation, real-time management, and integration, it shows the need to orchestrate not just a single platform or silo, but end to end across multiple layers, platforms, technologies, and vendors.

The value of an independent judge

I also really like the idea of specifically including an independent arbiter – the Cloud Auditor – that is empowered to “evaluate the services provided by a Cloud Provider in terms of security controls, privacy impact, performance, etc.” The need for an independent reviewer is already well overdue. Today, even cloud leaders like Amazon, WordPress, Salesforce, and Netflix can be down for hours with no reporting or explanation, and with no more payback than a sorry letter and few pennies in credit for time lost. They are also killing off any expectation of security, compliance, or privacy by hiding away fine print like the right to “access, retain, use and disclose your account information and your files … as [they] determine is necessary”. In this climate, we already desperately need an independent agent to adjudge the operations, performance and security of all cloud providers, especially public cloud providers.

However, the reference architecture is not all good, and some significant issues also stood out for me:

Security is a one-sided activity

The reference architecture hangs the responsibility for security almost entirely on the Cloud Provider, which is poor advice. Unlike cookies, security is not a ‘sometime’ food, and active participation in security cannot be attributed to any one actor or interaction. For example, two-factor authentication necessarily requires active participation by both service provider and service consumer. Cloud Auditors and Cloud Brokers also have significant responsibilities for security.

Everyone is a ‘Cloud Carrier’

The ‘Cloud Carrier’ actor essentially elevates all telcos to the role of ‘Cloud Carrier’ with no change in business model or technology. It also actually classifies cabs, couriers, and even the UPS as ‘Cloud Carriers’, as this actor includes any provider of “physical transport of storage media such as high-capacity hard drives.” A requirement for Cloud Providers to set SLAs with Cloud Carriers is especially unlikely for a public cloud, though it makes more sense in the context of a private cloud. It also leads to difficult questions of carrier interoperability, quality of service, traffic shaping, and even ‘Net neutrality.

Encryption is optional

Encryption is included as an optional (!) activity, which of itself is unacceptable for mission-critical enterprise applications. Even then it is ascribed to the Cloud Carrier. The idea sounds great – carriers provide “dedicated and encrypted connections” for the “connectivity and transport of cloud services.” However, it is unrealistic for carriers to implement interoperable encryption for ‘cloud traffic’ (whatever that is). It also forgoes the current, quite logical, de facto standard – encryption directly between the provider and the consumer, regardless of carrier. Again, much of this architecture seems to be more directed at private cloud networks, rather than public networks including the Internet.

Privacy is not having to say you’re sorry

Privacy is included as a single line item without much meat on the bone:

Protect the assured, proper, and consistent collection, processing, communication, use and disposition of personal and personally identifiable information (PII) information on the cloud.

This is so neutral as to be unhelpful. Sharing personal data with advertisers, handing over corporate data to warrantless investigations, or even selling your customer database on eBay, may all be ‘assured, proper, and consistent’ according to some so-called ‘privacy policies’. The document does allow the Cloud Auditor to “evaluate the services provided by a cloud provider in terms of … privacy impact,” but beyond this it has no advice on what privacy actually means. Perhaps this is asking too much of a high-level document, but personal privacy and data loss prevention are critical issues in cloud computing. From controversy over Facebook’s exposure of personal details to cloud providers cutting off legitimate businesses, there is significant concern over privacy. I expected more prescriptive advice, rather than a neutral academic definition, especially from a public body setting policy for the IRS, the Pentagon, Department of Social Security, and other sensitive departments.

Management is entirely a provider activity

NIST attributes cloud management – including security and service management – entirely to the cloud provider. This is rare (if it exists at all) among public cloud providers today, and is unlikely to ever be acceptable for most enterprises. With providers doing all the management the fox is watching the hen house. Consumers will require at least some participation. We learned that it is bad to give total control to third-party providers when we did things like IT outsourcing. Just as with cloud computing itself, the majority of enterprises will probably always want a hybrid model for cloud management.

The bottom line

I do see this as a very useful document. It is really quite good – as far as it goes. It is also a very important document – for what it gets right, for what it gets wrong, and for where it comes from, as NIST is helping to shape cloud standards for the world’s largest consumer of information technology. It is far from perfect, and I believe has some truly fundamental flaws, but it is only a Version 1, and who among us has delivered a perfect product on the first release?

So ultimately, it is good enough for now, but I am very much looking forward to the ongoing development of this document. To quote Raymond Umerley (@SecJitsu):

'Close Enough for Government Work'