You may as well look in a crystal ball!

Welcome to IT prediction season! Again, I am inspired to throw my exceedingly fallible hat into the ring with my predictions, specifically for virtualization and cloud. I seem to have had a decent run of predictions last year, but I claim more luck than credit. I still think predictions are a mug’s game, and continue to eschew both the importance and reliability of predictions.

That said, here are my predictions for 2012:

1. Brands May Come and Go – But No Technology Will Die

Not only are we not living in a ‘post-PC’ world, we are not even living in a ‘post-mainframe’ world! Cloud will not kill data centers, virtual will not kill physical, tablets will not kill PCs, Mac will not kill Windows, Android will not kill iOS, streaming will not kill DVDs. The technology pie is growing, our choices are expanding, and almost every slice is getting bigger. So be prepared to manage an ever-increasing selection of technologies across public and private boundaries.

2. Hybrid IT Will Be ‘The Next Big Thing’

‘Hybrid cloud’ was soooo 2011! In this new world of choices, business will expect hybrid IT: a combination of on-site and off-site; cloud and legacy; private and public; physical and virtual; social and secure; enterprise and consumer; desktop and server; mobile and static. Business will also expect IT to make them work together, whether IT owns the service or not. IT must act as a trusted advisor, as a service broker, and as quality assurance for this brave new world of complex Hybrid IT.

3. Service Quality Will Be IT’s Responsibility Again

As hybrid IT proliferates, business owners will (again) realize they do not want to manage technology; they just want it to work. In 2012, end users will increasingly expect IT to take responsibility for service quality, regardless of who is buying, selling, or delivering that service. IT will need to eliminate the blind spots in hybrid IT, actively support an explosion of devices, deal with complex cross-boundary services, and find a way to deliver a 360-degree service assurance across all facets of end-user experience.

4. Public Cloud Adoption Will Slow

Given the results of this year’s Longhaus research from Australia – an early adopter market and a bellwether for business technology – I suspect the rest of the world is in for a slowdown of public cloud adoption. Issues (perceived or real) with security, compliance, service quality, skills, staffing, complexity, and good old politics will all put the brakes on. Whether ‘cloud stall’ will be as pronounced as ‘virtual stall’ is unsure, but 2012 will see a marked slowdown in public cloud adoption.

5. Public Cloud ‘Gets’ Security

Sad but true – many (most?) enterprise decision-makers still do not trust public cloud. In 2012, IT must do a better job of deploying and explaining cloud security – and I believe we will! In 2012, CIOs will see security as less of a barrier to cloud adoption as organizations adopt more and better cloud-oriented security solutions – including solutions designed for complex hybrid cloud services, as well as solutions that are delivered through the cloud with easily-consumed Security SaaS options.

6. Big Iron is Back – Part I

No, mainframe is still not dead. On the contrary, 2012 will see the rise of the mainframe as a *gasp* cloud platform. Massively scalable, hosting critical (and underutilized) ‘big data’, capable of running complex cloud workloads on a variety of architectures (z/OS, Linux, UNIX, Windows), mainframe is really an obvious cloud platform. It will not replace commodity clouds, but large enterprises and governments especially will leverage their investments and bring big iron into their cloud mix.

7. Cloud Gets Heterogeneous

Not only will mainframe become part of the cloud landscape, but public cloud providers will also start to offer UNIX and maybe even other non-x86 platforms. I have recently seen this in action (CA did it internally years ago), and most large enterprises are heavily dependent on heterogeneous systems for their mission-critical applications. Despite the common myth that cloud == commodity servers, heterogeneous servers will start to become more available for large enterprise deployments.

8. Big Iron is Back – Part II

Big iron concepts of integrated compute, network, and storage are resurgent – but this is not your grandpa’s mainframe. Deployment of integrated fabrics like Cisco UCS and VCE Vblock will accelerate rapidly in 2012 as IT changes the way it thinks about integrated infrastructure for virtualization and cloud – and realizes how amazing these integrated boxes are for diverse, dynamic, high-volume workloads like desktop virtualization, pop-up data centers, and cloudbursting.

 9. ‘Grown-up’ Cloud Service Management Comes To The Forefront

In 2011, the NIST Cloud Reference Architecture devoted a whole section to ‘Cloud Service Management’, and IT started to talk about ‘grown-up’ disciplines – planning, budgeting, performance, asset, inventory, service levels, audit, etc. In 2012, even ‘commodity’ cloud vendors will finally take cloud management seriously, as enterprises and governments demand these disciplines – and smaller providers differentiate on service and security, not just price.

10. Virtualization Management Becomes Irrelevant

In January 2009 I predicted, “in 3-5 years … niche [Virtual System Management] vendors will no longer survive, as virtualization becomes a core part of the enterprise compute fabric.” Three years later this trend has definitely started, and will accelerate in 2012 as IT turns instead to hybrid IT management, recognizing that silos of standalone virtualization management is a costly and inefficient burden. Maybe 2012 is not the end of Virtualization Management, but it is going to be the start of the demise.

—

So that is my punt on 2012. I have no idea whether they will come true, but they seem to make sense to me. Again, if you are reading this in December 2012, please feel free to e-mail me and let me know how I went. I won’t be surprised either way.

[this post was originally published at VMblog.com]

I had a great time at VMworld 2011 Las Vegas this year. As I predicted in my last blog post, I met with loads of amazing people – too many to list out here, let alone in 140 on Twitter! I also saw some great technology in the solutions exchange, dropped in on some fascinating sessions, and of course enjoyed some excellent meals, drinks, and parties!

I was also very pleased to present on Extending the Value of Your VMware Solutions to Design, Deliver and Maintain Reliable, Mission-critical Virtualization and Cloud Services. I certainly was not there to ‘pitch’ any CA Technologies products or solutions (after all, I know that no one wants a sales pitch at a tradeshow like VMworld). Instead, I tried to provide strategic advice to the audience on how to look at their migration to cloud, and especially how to extend VMware’s excellent virtualization and cloud technologies.

My VMworld 2011 Las Vegas Presentation Agenda

With a smattering of additional tips and content from ‘Visible Ops – Private Cloud: From Virtualization to Private Cloud in 4 Practical Steps’, I talked about:

• how to match services with the right cloud using project and portfolio analysis based on models from Visible Ops – Private Cloud, a CA Technologies quadrant framework, Forrester Research’s Evaluating Application Fit With Cloud model, and Freeform Dynamics’ model from Applied Cloud Computing: A practical guide to identifying the potential in your environment
• how to think about your service portfolio, whether considering migrating existing services to a private VMware cloud, building new services on a public VMware cloud, dealing with business users who buy into 3^rd-party cloud themselves, or even services that you may never migrate to the cloud
• how to leverage VMware to deliver both evolutionary cloud models built with virtualization, optimization, automation, orchestration, and dynamic IT; and with revolutionary models that deliver exponential benefits with a virtual business service, built on a virtual service fabric
• how to integrate complex service workflows, skillsets, and technologies, as well as incorporating NIST best practices including cloud service management and service-aware end-to-end application assurance to continually improve service quality, predictability, and costs
• how to apply critical security disciplines including Identity Management & Provisioning, Identity Federation & Single Sign-On, Web Access Management, Privileged User Management, Identity Compliance, and User activity reporting, whether to, from or for the cloud
• how to approach cloud as a transformation opportunity, so you don’t just do the same things in different ways, but fundamentally transform business and IT, delivering a ‘cloud of clouds’ with a broad technology ecosystem stocked with key VMware partners (like CA Technologies!)

You can check out my slides at the CA.com communities site, or over at SlideShare.

Overall, my session seemed to be very well received. A lot of people came up to me there and afterwards and told me how much they enjoyed my presentation, and how useful it was for them. I also enjoyed a great set of questions from the attendees immediately after the session. In fact, we were chatting so much we had to be ushered out so the next session could start.

Immediately afterwards I headed down to the CA Technologies booth, and really enjoyed talking with various practitioners and others at the book signing for ‘Visible Ops – Private Cloud: From Virtualization to Private Cloud in 4 Practical Steps‘ afterwards (with co-authors Jeanne Morain and Kurt Milne). I even had a professor in IT from NYU ask for a copy of my book! Cool!

All in all, I had a great time, made new friends, enjoyed great food, and even managed to avoid the possible downsides of VMworld!

I hope VMware Europe Copenhagen will be just as good – and I hope to see you there!

BSMdigest

I have just started posting on BSM Digest on behalf of CA Technologies. You can read my first post there, Beyond BSM – Cloud Computing and the Next Generation of Service Management! In it, I talk about how cloud drives an urgent need for a more flexible and dynamic management practice based on automated service operations management.

Rather than adding yet more manual labor, or continuing to focus on low-level change and configuration management, cloud drives an urgent need for a more flexible and dynamic management practice based on automated service operations management. This is driving a new generation of service management that can assure quality business service in real-time, even as infrastructure and applications rapidly change, and which can trigger workflows across data center and cloud resources to rapidly remediate service quality and availability problems.

Read the full article: Beyond BSM – Cloud Computing and the Next Generation of Service Management at BSMdigest.

itWorld Canada

I had a great interview recently with Kathleen Lau for her feature article in IT World Canada, Sprawl or Stall: How’s your virtualization project going? where she looks at the causes behind two common problems plaguing virtualization deployments, and how IT departments can craft their strategies from the outset to avoid these hurdles.

It’s not unusual for IT departments to either slow down or stop a project mid-way in order to reorient themselves. But when it comes to virtualization, taking a breather for a necessary re-assessment is a sign that the IT organization is pulling back on its virtualization strategy, said Andi Mann, vice-president of virtualization product marketing with Islandia, N.Y.-based CA Technologies Inc.

Check the whole article here – Sprawl or Stall: How’s your virtualization project going?

SilconAngle TV

At EMC World 2011 recently, I had the pleasure of chatting with Stu Miniman, Analyst and Researcher at Wikibon, on the SilconAngle TV show about management, cloud and organizational transformation. You should definitely go to Wikibon.org to see more from Stu, and/or check out his blog. And do check out siliconANGLE TV as well – they cover many of of the major shows with great video coverage, interviews, etc.

Meanwhile, check out the video below:

Or go on over to siliconAngleTV and see Andi Mann at EMC World 2011.

Which is the bigger myth - public cloud cost savings or Lewis Carroll's Jabberwock?

There is a lot of hype around how much money you can save with public cloud.

Public cloud can be cheaper than on-premise IT or private cloud, especially for selected services and SMBs. However for large enterprises, while there are plenty of reasons to use public cloud, cost reduction is not always one of them.

Public cloud certainly has a low startup cost, but also a long ongoing cost. For all practical purposes, the ongoing cost is never-ending too. As long as you need it, you keep paying as much as you did on day one, without adding an asset to your books or depreciating your facilities investments.

Public cloud also rarely (ever?) provides the management, security, governance, or lifecycle solutions that add cost to internal IT either – not because public cloud does not need them, but because it eats into cloud providers’ profit margins. Indeed, the myth of low cost cloud is built directly on the equally false myth that public cloud doesn’t need management or discipline.

Better economies of scale are not assured either. Many large IT organizations have more and larger data centers than most, though perhaps not all, public cloud providers. Enterprise IT can be strikingly massive, with economies of scale that match or exceed even the largest public cloud providers.

Public cloud does not always lead to a substantial reduction in staff costs either. Simply sourcing infrastructure or software as a service does not let you disband your IT department entirely. After all, your databases still need DBAs; your applications still need performance monitoring; your users still need a Help Desk; regardless of where the services or systems are running.

This last issue was partly the basis for an early debunking of the ‘public cloud is cheaper’ myth in a McKinsey report titled ‘Clearing the Air on Cloud Computing’ (PDF) back in 2009. McKinsey found that IT labor cost savings with public cloud (IaaS in particular) are minimal, around 15% mainly in sysadmins. However, non-labor costs were actually much higher for public cloud. Overall, it showed a 144% higher cost of public cloud over internal IT, and concluded that, with the majority of cloud savings achievable in-house with virtualization, an investment in private infrastructure makes sense for larger organizations.

A lot has changed since 2009, and both platform and software clouds offer different benefits to infrastructure clouds. Still, many of the CIOs that my co-authors and I interviewed for our recent book ‘Visible Ops – Private Cloud: From Virtualization to Private Cloud in 4 Practical Steps‘ confirmed this from their own experience. These practitioners have done the calculations on their own real world IT budgets, and told us how they can deliver the same (or better) service internally at a 30% discount to public alternatives – in accordance with the Visible Ops – Private Cloud mantra of ‘faster, better, cheaper’:

For example, the larger organizations we interviewed are confident that they can offer their own public cloud-like services cheaper, better, and more securely than external service providers. As a result, many are working to create a self-service, low-touch model that can be used to spin up new computing resources.

In Visible Ops – Private Cloud, we also cited primary research conducted and published by Wikibon on the comparative cost of public cloud for large enterprises. It has a more detailed analysis of economies of scale, and found that only for low-priority applications in SMB environments will public cloud be more cost-effective than private cloud, and then by only 14%. By contrast, private cloud is more cost-effective for all large enterprise use cases, and for all mission-critical applications (even in SMBs), by up to 41% annually, primarily because:

1) The applications running in larger organizations demand more resilience, governance and control;

2) While small businesses will find public clouds attractive, large organizations will be able to create a private internal cloud that has many of the same business characteristics as a public cloud, but with much higher levels of control, security and availability; and

3) Achieving the same levels of resilience with public cloud infrastructure would be prohibitively expensive for larger organizations.

ASG has also published a short analysis of the comparative TCO of private and public clouds on their blog. This analysis has many built-in assumptions, but it does show a reasonable and defensible interpretation of costs in both environments. According to this analysis, over a 3-year period, private cloud operation (whether on owned or leased equipment) is around half the cost of public cloud:

This analysis shows a $513,295 cumulative cost savings over three years for the private cloud purchase versus the public cloud option [$1,092,188]. The private cloud lease saved $489,874 versus the public cloud option over the same three years.

Then there is James Staten of Forrester research, and a great report titled ‘The 3 Stages of Cloud Economics’. Unfortunately for many, the full report is subscriber-only, so I cannot extract it, but it is a great read (you can get a free taste over at James’ blog at Forrester.com on the stages of cloud economics). Suffice to say that the report starts out front and center with a whole section titled ‘BUSTING THE MYTH — CLOUD COMPUTING ALWAYS SAVES YOU MONEY’ [sic].

Of course, a realistic cost analysis depends on too many organization-specific factors to draw firm conclusions for any but specific use cases. However, for most enterprises evaluating cloud (public or private), an overly tight focus on cost reduction is misplaced at best. Given all the experience and data at hand, cost reduction is at best a possible outcome from public cloud, and even then only for some workloads, and only for some organizations.

Yet all of this ultimately misses the point. As James Staten says in his blog:

But this isn’t a debate worth having because it’s the exploration of the use cases where it does save you money that bares the real fruit. And it’s through this experience that you can start shifting your thinking from cost savings to revenue opportunities.

Clearly cost reduction is important when you can achieve it. Indeed, judicious use of cloud computing, public and private, for the right services, the right users, at the right time, certainly does deliver cost reductions. However, while cloud in general and public cloud in particular can save money, cost reduction is certainly not the only reason to use cloud, public or private; in my opinion, it is probably not even the best reason to use cloud.

The potential that cloud computing creates to drive technology flexibility, business agility, quality of experience, and service continuity (even despite multiple high-profile public cloud outages) far exceed the returns to large enterprises of hardware cost reduction. The real benefit is found in business gains like greater competitive advantage, faster time to market, and higher customer retention.

It is through these business factors – not mundane IT line items such as server purchase prices and administrator salaries – that cloud computing, both public and private, will drive the most significant and fundamental changes to the bottom line for most large enterprises.

The front cover of Visible Ops Private Cloud

Today is a very exciting day for me.

After many months of blood, sweat, and tears, today we are finally launching a brand new book that I co-wrote (alongside a couple of amazing people – Kurt Milne and Jeanne Morain), called Visible Ops Private Cloud: From Virtualization to Private Cloud in 4 Practical Steps.

I am certainly excited because this is my very first ‘real’ book – a hard-copy print volume (Kindle and PDF versions will be available soon too), available for purchase on Amazon.com and ITPI.org, with a real ISBN and everything!

But that is not the only reason.

I am excited to be working with the book’s publisher, the IT Process Institute. The ITPI is a renowned independent research organization “that exists to support the membership of IT operations, security, and audit professionals” with a  mission “to identify practices that are proven to improve the performance of IT organizations.”

In addition, I am thrilled to have worked with experienced professionals like Jeanne and Kurt . Together, we have over 60 years experience in systems management, automation, virtualization, and cloud with some of the best and brightest vendors including VMware, CA Technologies, BMC Software, InstallFree, Thinstall and more.

Moreover, I am proud that Visible Ops Private Cloud is based on over 30 in-depth interviews with actual practitioners – managers, architects, CIOs, administrators, and others working in healthcare, finance, service providers, education, manufacturing, hi-tech, and other enterprise IT organizations, who have successfully implemented private cloud solutions.

Most especially I am excited because with this combination, we have created something uniquely valuable in Visible Ops Private Cloud.

If you are aware of the famous Visible Ops series of books from the IT Process Institute (ITPI), you will instantly understand how proud I am of this effort, and my association with these great texts and their publisher. You will know that these books are concise and easy to read (Visible Ops Private Cloud is just over 100 pages in a paperback-sized book), providing “simple, practical, step-by-step advice on how to quickly gain the biggest benefit from proven best practices.” You will know that they are “the ideal way to get an IT organization to rally behind a simple approach to operational excellence.”

You will also know that this is a book for ‘doers’, not just ‘thinkers’. Indeed, in keeping with the mantra of the Visible Ops series, Visible Ops Private Cloud aims to provide prescriptive, actionable guidance on how to drive cloud success by turning your virtualization project into an enterprise private cloud strategy, based on best practices from other professionals and organizations.

This is why in Visible Ops Private Cloud we broke down the evolution from virtualization to cloud into four clear, pragmatic phases:

• Phase 1: Cut through cloud clutter – plan and communicate objectives, manage initial proof of concept efforts, and develop competency road maps.
• Phase 2: Design services, not systems – design business optimized cloud services, enable one-touch service ordering, and implement a repeatable approach for build and deploy.
• Phase 3: Orchestrate and optimize resources – update monitoring and alerting, deploy a policy engine to codify response, automate resource changes and workload moves.
• Phase 4: Align and accelerate business results – complete transition to a resource rental model, reshape consumption behavior, and streamline response to changing business needs.

While we were trying to provide useful content for an audience anywhere from a sysadmin up to CIO, the real sweet spot will be those practitioners who are responsible for the initial design, ongoing deployment, and ultimate success of their organization’s private cloud strategy – the ops manager, sysadmin team lead, enterprise architect, CTO office, development manager, virtualization lead, or the virtualization/cloud project manager.

Mainly we tried to recognize where these people were really up to with their cloud strategies, and to provide useful and actionable advice to help them advance their progress. So in this book you will not read much about how to optimize hypervisor configuration settings, or how to schedule a VM backup; nor will you see much high-level exploration of elementary cloud definitions or the conceptual ‘journey to the cloud’.

What you will see is a concise run-down of the situations in which you might find yourself; the defined steps you need to take to make and show progress; how to put this advice into action across the affected people, processes, and technologies; and some real-life war stories highlighting how to avoid specific problems, and explaining critical issues you will need to understand.

That said, we certainly hope that Visible Ops Private Cloud will provide an excellent resource not just for these strong practitioners’ own use. We also expect this book to be a valuable resource they can use to educate their CIOs on the tactical realities of building a private cloud, and to educate their more focused hands-on colleagues on the important strategic business and IT goals that their work is helping to accomplish.

Perhaps the most important part of Visible Ops Private Cloud is that our personal experience is not the sole factor – or even necessarily the main factor – in how good the content in this book turned out. Despite the authors’ sometime involvement with various software vendors, this is truly an independent effort, without a single brand or product name mentioned throughout.

On the contrary, the real stars throughout the book are the practitioners – the administrators, sysadmins, programmers, operators, CIOs, enterprise architects, and others – and their experiences in virtualization and cloud. Their insights into what makes a private cloud successful – and some of their war stories too – come through loud and clear. Their stories ensure Visible Ops Private Cloud delivers a very realistic, pragmatic, and independent investigation based on real-life implementations.

So how good is Visible Ops Private Cloud really? Well, you can do a search for it, and see what other people are saying (I will try to update this blog with some links as they come out).

[Edit: Here are some of the reviews that have come through so far:]

• By The Bell: Book review: Visible Ops Private Cloud: from virtualization to private cloud in 4 practical steps

You can go over to Amazon.com and check out the reviews (there are already a couple there, and hopefully more to come). You can read the back cover and see some of the endorsements that came in after we sent out some preview copies:

Visible Ops Private Cloud back cover quotes

Or you can simply order a copy from Amazon or direct from the ITPI and read it yourself. If you do – and I really hope you will – please let me know. I would love to hear what you thought.

More than four reasons count towards VM stall

I recently saw a great article in IT World Canada titled “Virtual stall: What it is and why you have it,” written by Jay Litkey, that took up my idea of VM stall, which I first came up with in my blog from May ‘Is “VM Stall” the Next Big Virtualization Challenge?‘.

Though they barely acknowledge my blog as their inspiration (and as a competitor to CA Technologies – my employer – why would they?), it seems Jay and his team have wholeheartedly taken up my concern with VM stall, and not just in the IT World Canada article. Marketing lead David Lynch was quoted on the topic in a post by Bruce Hoard of Virtualization Review, and in a recent Tech Target article on ‘ISV stall’. Several posts on their corporate blog also address the issue as if it was their own baby.

In my past life at EMA, I have spoken with both Jay and David a number of times, and had a lot of time for what they were doing in the management space. For a small startup with limited resources, it is great that they can take the time to pick up my idea and run with it.

The IT World Canada article is really worthwhile, because it zeroes in on some important concepts. It helps to expand the thought around VM stall, and specifically on a couple of additional causes, as it notes:

Virtual stall has four main causes:

• Scalability issues:  A single IT team often finds it difficult to scale beyond the 25-30 per cent penetration range. This is due to the combination of lack of automation and reporting in virtualization management tools, creating time-consuming manual processes that are a particular problem when there is a lack of experienced and trained staff.
• Management issues: The data centre is not a place that can be managed manually; there are too many elements to be checked, and too many independencies [sic]. And, while there are levels of automation built into the virtualization platform, they can be difficult to define and implement. The lack of automated monitoring, alerting and control becomes more and more of a problem as the overall level of virtualization in the data centre increases.
• Process issues:  Enterprise virtualization impacts a wide range of existing data centre processes, all of which need to be modified, replaced, or augmented. As long as the virtual environments are small and self-contained, these processes can be manipulated or ignored. But as the environment grows, it reaches a point when they have to be dealt with before real efficiencies can be reached. The more “process-mature” an organization is, the more quickly this point is reached.
• Co-ordination issues: Virtualization crosses multiple silos and ultimately requires a level of co-operation and integration that is impossible to achieve with the traditional silo management structure. In addition, the first workloads to be virtualized tend to be less critical ones.  However, as environments grow, higher-risk, higher-impact services are virtualized. These tend to have more stakeholders, more politics, more distributed infrastructures, and a greater cost of failure and downtime. Consequently, they require more coordination.

This is great insight, and offers a number of important causes. However, I don’t think it is reasonable to say there are just “four main causes.” Not to pick on Jay, as it is probably just unfortunate phrasing, but I think it is important to see that the issues of VM stall are much more varied, complex, and numerous.

I am not entirely without fault either. To start with, when I first identified the issue of VM stall in my blog post back in May, I said that “I see many possible causes for VM stall,” but like Jay I only identified four examples. As Jay recounts in his analysis, I saw scalability and manageability as key issues; but unlike Jay, I chose to highlight risk aversion and resourcing as two more of my examples.

However, even these six are just a part of the problem. As I said when I spoke with my great mate (and one of the industry’s great virtualization gurus, observers, and commentators), David Marshall of Hyper9 and InfoWorld in his article, “VM stall: Breaking through the second phase of virtualization“:

“… many organizations strike a ‘perfect storm’ of challenges that slows their virtualization rollout, or stops it entirely. Some causes at this stage include greater complexity of services and applications, higher demand on scarce virtualization skills, limited visibility into a growing deployment, increasingly heterogeneous systems, and greater resistance from risk-averse application owners and recalcitrant application vendors.”

In the same article, David spoke with Dave Bartoletti, formerly of automation vendor Enigmatec and now a leading light showing the way through the virtualization darkness with research and advisory analyst firm, the Taneja Group:

“The second wave of issues is always harder when a core technology matures. Server virtualization essentially paid for itself in CAPEX savings, but when we virtualize Tier 1 business-critical applications, or user desktops, CAPEX savings take a backseat to application performance and IT efficiency, and this is why we’re stalling.”

My former editor at Tech Target and another keen virtualization observer, Colin Steele, highlighted another core element of VM stall, in his article “ISV stall makes virtualizing applications a challenge“:

By now, the benefits of virtualizing applications are clear, but the goal of 100% virtualization remains elusive. One reason is that some independent software vendors (ISVs) don’t support their server-based applications — databases, telecom apps, healthcare programs, etc. — on virtual servers.

Moreover, I talk a lot with customers about their real world concerns, so I can quickly pinpoint many other causes. They talk to me about issues like vendor licensing, facilities constraints, capacity blindness, service prioritization, deployment costs, line-of-business resistance, internal politics, a lack of skills, and even senior management resistance.

In fact, last week at CA Expo in Australia, I talked with CA Technologies customers about seven significant issues in virtualization that are contributing to (among other things) VM stall, as you can see from one of the slides from my presentation:

Virtualization is not clear sailing - from CA Expo Australia

(You can see the whole deck at the CA Expo site)

To be fair to Jay and his team, other posts on his corporate blog agree with me, citing issues like mission-critical apps, management skepticism, bureaucracy, poor project vetting, and more.

I am really glad to see my thoughts around VM stall have captured the imagination of the market. Thanks to Jay for taking this up, and to his team for joining me and CA Technologies in raising awareness of issues causing VM stall.

However, I think we all need to be careful about being categorical about VM stall. It is important to be clear that VM stall – like most enterprise IT issues, and indeed most organizations – is both complex and varied, so trying to categorically define four (or six, or seven, or really any number) of causes for VM stall is underestimating this important problem.

But if we can all contribute new ideas to the community, we will all learn more, and our enterprise customers will benefit from our combined wisdom.

Most devops discussions are missing the target

I am hearing a lot about the rise of a concept called ‘devops’ – a mashup of ‘development’ and ‘operations’. I am not at all an expert in this area, but from what I can tell, devops is aimed at streamlining rapidly iterative application delivery to allow for greater development and business agility. Devops aims to achieve this by breaking down the barriers – human, process, and technology – between application development and system operations.

Interestingly, the concept is new enough that, as I write this, there is not even an entry for it in Wikipedia yet. I did find a blog by Damon Edwards (on Twitter – @damonedwards) very useful though, as he explains the age-old disconnects between application developers ‘throwing software over the wall’, and ops who are painfully resistant to change. James Urquhart (@jamesurquhart ) blogged very recently on the concept too , and again provided some very helpful content. Conversing online with them and others also helped me to formulate some more concrete ideas about devops – or at least some more concrete questions.

My interest was especially piqued when I understood how closely devops is connected to virtualization, cloud, and automation – my core interests:

• Cloud – Devops has antecedents in ‘rogue’ developers (or developers from smaller shops) using cloud resources (IaaS, PaaS) for new projects, and will benefit greatly from cloud-based development and deployment, as cloud providers do not impose the restrictions of internal change-averse ops teams, and developers can essentially manage their own ops requirements instead.
• Virtualization – In-house devops (which needs more heavy lifting) is greatly assisted by virtualization, as virtual machines become the new base unit for application packaging, avoiding application rollout failures caused by incompatibility between the test and production environments (hardware, OS, middleware, etc.).
• Automation – In-house devops is also greatly facilitated by automation, which can use standard workflows to automatically provision and configure these complete application VMs, as well as backup and restore VMs, allowing complex composite application deployment and rollback at the click of a mouse.

Clearly devops has many very attractive outcomes – drive agile business, reduce delays, smooth application releases, deliver value faster. It is a very seductive idea. Who wouldn’t want it?

However, most of the writings I see about devops are really about dev, not ops. As a result, they don’t really capture the whole story of the application lifecycle. They justify devops as an antidote to the problems that ops are causing – slowing down release cycles, imposing arbitrary rules, screwing up deployments, killing developer productivity, hacking manual scripts and configs, stopping the business from being agile – but fail to recognize both the failings of developers that contribute to the problems, and the role of operations in delivering critical business outcomes during the application delivery lifecycle.

On the contrary, discussions mainly focus on how developers can sideline or change operations, positioning devops as the lone hero in the battle against inefficiency, as application developers fix all the problems (!) by controlling or automating key release management operations like provisioning, deployment, integration, patching, and software update. Meanwhile, ops are marginalised, along with their timesinks and roadblocks, satisfying the needs of an agile and rapidly changing business.

See – seductive, isn’t it?

Yet this seems to me (as a former op) fundamentally flawed, a development-centric neologism based on an incomplete understanding of the real purpose and role of IT operations, or of operations’ history in the development of ‘agile’ IT.

The way I see it, devops misses that target on how IT ops serve business needs too, and seems to gloss over ‘coal face’ realities like:

• Who handles ongoing support, especially software update for the unrestrained sprawl of non-standard systems and components.
• Who ensures each new application doesn’t interfere with existing and especially legacy systems (and networks, storage, etc.)?
• Who handles integration with common production systems that cannot be encapsulated in a VM, like storage arrays (NAS, SAN), networking fabrics, facilities, etc.
• Who handles impact analysis, change control and rollback planning to ensure deployment risk is understood and mitigated?
• Who is responsible for cost containment and asset rationalization, when devops keeps rolling out new systems and applications?
• Who ensures reporting, compliance, data updates, log maintenance, Db administration, etc. are built into the applications, and integrated with standard management tools?
• Who will assure functional isolation, role-based access controls, change auditing, event management, and configuration control to secure applications, data, and compliance?

Because, if you have ever worked with both ops and apps, you know it is not going to be apps.

Now, in defence of devops, I am sure it is being implemented and conferring major benefits, especially in small organizations with little IT management discipline. I am sure the supporters of devops have some positive goals in mind too. What’s more, it is addressing a very real problem – ops really should spend more time on better processes and controls than in ‘daily deployment muck’.

However, devops should be a two-way street. As a former op, I know that the apps team have to pull their weight too, by addressing gaps like:

• Including ops during the design process, so applications are built to work with standard ops tools
• Taking ops input on deployment, so applications will go in cleanly without disrupting other users
• Working with ops on capacity and scalability requirements, so they can keep supporting it when it grows
• Implementing ops’ critical needs for logging, isolation, identity management, configuration needs, and secure interfaces so the app can be secure and compliant
• Giving ops some advance insight into applications, especially during test and QA, so they can start to prepare for them before they come over the wall
• Allowing ops to contribute to better application design, deployment, and management; that ops can do more for the release cycle and ongoing management than just ‘manipulating APIs’

See, ops do enable business – and agile business at that – by ensuring that new applications coming into an existing complex environment are safe, secure, reliable, integrated, and responsive, regardless of how complex IT is, or how many moving parts there are. Devops seems to miss this important detail.

So I am sceptical of how devops will work in large, well-run IT environments with important and necessary operational controls, especially the > 60% of organizations that are committed to ITIL best practices (like formal and integrated management of change, configuration, release, assets, etc.).

After all, ‘agile’ does not magically obviate the need to identify and prevent bad changes, to reject apps that breach operational compliance, to ensure each new application adheres to standards, or to prevent uncontrolled sprawl of heterogeneous software.

I still have a lot to think about on this topic, and am trying to keep an open mind. But my best guess right now is that, for enterprises at least, devops either will not take hold or will not last. It seems most likely to be instead, at best, a transitory state on the path to a ‘new normal’. As with all ‘revolutions’, it has started outside IT ops, yet I expect will eventually co-opt and migrate wholly to operations in some form. Once the revolutionaries in development understand how many business needs besides agility actually require  routine, process, management, and controls, they will back away from devops the same way they backed away from ownership in other IT revolutions – like the deployment of mini computers, desktops, and web applications.

If it does turn out this way – don’t worry. Operations will again dutifully take the reins, and clean up the mess that devops will leave behind. Because that is what ops do – they manage what they are given, and keep the business running, regardless of the mess that gets thrown over the wall at them.

In any case, whether devops takes root or not, hopefully we will all learn something about cooperation, automation, agility, and control. Because all stakeholders in the devops discussion – development, operations, and business owners – could benefit from that.

Effort vs. Payback is an Everyday Business IT Decision

I read recently a good blog post from Thomas Bittman (@tombitt) of Gartner Group, about how sometimes close enough is good enough. Talking specifically about private cloud, he talked about how an ‘imperfect’ cloud deployment – one that does not have all five essential characteristics, for example – might be enough for some organizations.

I especially appreciated how he highlighted some very specific, real-world examples to sustain his advice. As he shows, sometimes you don’t need a ’100%’ implementation, and for very good business reasons.

Not every IT organization needs a fully self-service interface, and many smaller organizations see no value in usage metering. They simply want to deliver services faster. For them, a 70% private cloud is absolutely good enough … it all comes down to business requirements, return on investment, and future strategy. How far you go is your decision.

via Driving for Imperfection With Your Private Cloud.

If you haven’t seen it yet, you should. It’s a quick read, only 4 paragraphs and less than 300 words. Go ahead. I’ll still be here when you get back.

The theme is very similar to something I wrote in a research report for EMA, ‘The Responsible Cloud‘, also on cloud computing. Regarding the NIST definition of cloud, I cautioned against dogmatic interpretations of cloud computing, and the notion that a ‘real’ cloud must necessarily have all of the essential characteristics, or fit some specific deployment model. Flexibility is key, I advised, and delivering on key business requirements is more important than definitions.

Two other things happened this week that made me think about this in different ways:

• An internal session at CA reviewing some customer-facing materials. All attendees agreed – we can’t preach unattainable dogma; we need to deal with specific requirements and partial deployments, as well as broad requirements that come from  ’100%’ implementations.
• A group discussion on LinkedIn, where an IT practitioner wanted advice on building a small private cloud. He was soon inundated with an unrealistic list of requirements, from hypervisor features to management disciplines, that he *must* have to build a ’100%’ cloud.

The similar inferences in three otherwise unrelated conversations started me thinking more broadly about ’100% adoption’. It IT, as in life, you never really need a Rolls Royce. You can aspire to the quality, appreciate its refinement, and in some cases you may be fortunate enough to actually enjoy it, but there is a point where it simply doesn’t make sense to pursue that level of luxury. Mostly you can get away with a Ford. Sometimes you can even make do with a second-hand Lada.

The same Pareto-like principle applies roughly throughout IT (much to the annoyance of just about every security pro I have ever met) – although the actual ratio may vary wildly, you can often get most of the benefit from less than a ’100%’ implementation.

The phrase that sprang to mind for me was the same conclusion that I published elsewhere in the Responsible Cloud report, and the same notion that many IT pros live by, day in and day out:

It is important to look for opportunities, and do what makes sense

This should not just apply to cloud computing, but across all of IT.

Take, as another example, adherence to the IT Infrastructure Library (ITIL). Now, ITIL is a great framework, and an increasingly definitive reference for best practices in IT management. Data I have seen suggests as many as 60% of all IT organizations are committed to ITIL, and that implementation of ITIL (whatever that actually means) results in measurable and specific benefits in IT costs, staff and server efficiency, operational maturity, and more.

However, I also hear and read somewhat justified rants about how “ITIL just doesn’t work … ITIL is more 1960s than 2010 … it’s useless.” Yet the truth is, as so often, somewhere in the middle. In this too enterprises can definitely benefit from avoiding the dogmatic application of every single prescription. The same is true for other standards such as COBIT and ISO, or prescriptions from standards groups like the DMTF or NIST. All can deliver significant benefits with less than a 100% implementation.

It also applies in internal adoption of standard operating environment (SOE) components, like making singular (and often binding) choices between, for example:

• VMware vs. Hyper-V vs. Xen
• HP vs. Cisco vs. IBM
• HDS vs. NetApp vs. EMC
• Windows vs. Linux vs. UNIX
• iPhone vs. WinMo vs. Blackberry
• Solution suites vs. point products
• Mainframe vs. Commodity
• Physical vs. virtual vs. cloud

In all these cases and more, although standardization can have specific benefits, the greatest benefit to the enterprise does not always accrue from making an exclusionary choice; from committing to a 100% implementation. Most IT practitioners know that heterogeneity is the new standard – whether intuitively or grudgingly. They know that sometimes the best – or at least necessary – outcomes arise from providing multiple choices, fit to support multiple use cases.

Of course some areas are less flexible. You cannot, for example, pick and choose which parts of PCI, HIPAA, or Sarbanes-Oxley compliance would work best for you. Perhaps ‘close’ only matters in horseshoes and hand grenades, but for sure it doesn’t matter in legal compliance.

However, where possible, IT – practitioners, consultants, vendors, and analysts – need to stay away from dogma. We must avoid making any architecture, maturity model, or industry standard a religious ‘all or none’ battle. Important though they may be, these are not religious battles. These are IT decisions. Moreover, these are business decisions. So we need to keep the business goals in mind, and realize that sometimes a ’100%’ implementation simply does not make sense.

For those who have asked, and others who may care to know – the rumours are indeed true. After many happy years leading the fantastic systems and storage management team at one of the very best IT industry analyst and consulting firms, Enterprise Management Associates (EMA), I have moved on to take up an exciting new opportunity.

As of Wednesday this week (2/24), I am now at one of the very best IT management software vendors, CA Inc., where I am leading product marketing for their — our — virtualization management solutions.

In many ways, this was an incredibly difficult decision. EMA is a truly excellent place to work, and the role of an industry analyst was fascinating and fulfilling. The people I worked with and for are some of the best minds in IT – always intellectually stimulating, and straight-out fun to be with. It was truly my privilege to get to know them all, and especially to help my clients and my team to be successful.

Yet this was also one of the easiest decisions I have made. I believe both virtualization and management deliver incredible IT and business benefits, and as virtualization becomes increasingly ubiquitous, management of virtual systems becomes increasingly critical. I have long considered CA a leader in physical and virtual systems management, and believe CA has a great opportunity to extend its leadership in virtualization management, by helping even more IT and business people to be even more successful. As a part of  CA now, I can not only be a part of that opportunity, but can be a significant author of that success.

Moreover, it allows me to indulge my passion for technology and my expertise in marketing in an in-depth, direct, and focused way, rather than the broad, ancillary, and essentially academic role of an industry analyst. I will be able to work directly with some the biggest and most successful companies and technologies, not just in the US, but around the globe. Plus, like EMA, CA also has some incredible minds who are some of the most fun people to hang out with too.

While some will see this a move (back) to ‘the dark side’, I have always considered analysts and vendors to be two sides of the same coin – helping IT to deliver business services in more effective and efficient ways. While some may say that I have ‘sold out’ my integrity as an analyst, I have always considered my integrity to be a core and consistent value — and a non-negotiable one — regardless of my employer. While some may think I can no longer champion the best interests of enterprise IT like I did while I was an analyst, I believe the best software companies, and their best people, succeed and thrive specifically because they do exactly that.

As for this blog (and my Twitter feed), all my reasons for blogging and tweeting, and what I hope to achieve (both personally and professionally) with social media, are still the same as they were when I started. I therefore intend to continue writing and posting my personal opinions and insights about technology and other areas that interest me. After all, the areas I work with haven’t really changed, so I am still going to post about virtualization, systems management, data center operations, and cloud computing.

So although I cannot help but be informed by my current position and experience, my goal is to keep posting interesting and informed ideas, regardless of my employer. No doubt some people will stop reading — which is fine — but I still hope you will keep inspiring, contributing to, reading, commenting on, and arguing about these part-time musings of a full-time technologist.