More than four reasons count towards VM stall

I recently saw a great article in IT World Canada titled “Virtual stall: What it is and why you have it,” written by Jay Litkey, that took up my idea of VM stall, which I first came up with in my blog from May ‘Is “VM Stall” the Next Big Virtualization Challenge?‘.

Though they barely acknowledge my blog as their inspiration (and as a competitor to CA Technologies – my employer – why would they?), it seems Jay and his team have wholeheartedly taken up my concern with VM stall, and not just in the IT World Canada article. Marketing lead David Lynch was quoted on the topic in a post by Bruce Hoard of Virtualization Review, and in a recent Tech Target article on ‘ISV stall’. Several posts on their corporate blog also address the issue as if it was their own baby.

In my past life at EMA, I have spoken with both Jay and David a number of times, and had a lot of time for what they were doing in the management space. For a small startup with limited resources, it is great that they can take the time to pick up my idea and run with it.

The IT World Canada article is really worthwhile, because it zeroes in on some important concepts. It helps to expand the thought around VM stall, and specifically on a couple of additional causes, as it notes:

Virtual stall has four main causes:

• Scalability issues:  A single IT team often finds it difficult to scale beyond the 25-30 per cent penetration range. This is due to the combination of lack of automation and reporting in virtualization management tools, creating time-consuming manual processes that are a particular problem when there is a lack of experienced and trained staff.
• Management issues: The data centre is not a place that can be managed manually; there are too many elements to be checked, and too many independencies [sic]. And, while there are levels of automation built into the virtualization platform, they can be difficult to define and implement. The lack of automated monitoring, alerting and control becomes more and more of a problem as the overall level of virtualization in the data centre increases.
• Process issues:  Enterprise virtualization impacts a wide range of existing data centre processes, all of which need to be modified, replaced, or augmented. As long as the virtual environments are small and self-contained, these processes can be manipulated or ignored. But as the environment grows, it reaches a point when they have to be dealt with before real efficiencies can be reached. The more “process-mature” an organization is, the more quickly this point is reached.
• Co-ordination issues: Virtualization crosses multiple silos and ultimately requires a level of co-operation and integration that is impossible to achieve with the traditional silo management structure. In addition, the first workloads to be virtualized tend to be less critical ones.  However, as environments grow, higher-risk, higher-impact services are virtualized. These tend to have more stakeholders, more politics, more distributed infrastructures, and a greater cost of failure and downtime. Consequently, they require more coordination.

This is great insight, and offers a number of important causes. However, I don’t think it is reasonable to say there are just “four main causes.” Not to pick on Jay, as it is probably just unfortunate phrasing, but I think it is important to see that the issues of VM stall are much more varied, complex, and numerous.

I am not entirely without fault either. To start with, when I first identified the issue of VM stall in my blog post back in May, I said that “I see many possible causes for VM stall,” but like Jay I only identified four examples. As Jay recounts in his analysis, I saw scalability and manageability as key issues; but unlike Jay, I chose to highlight risk aversion and resourcing as two more of my examples.

However, even these six are just a part of the problem. As I said when I spoke with my great mate (and one of the industry’s great virtualization gurus, observers, and commentators), David Marshall of Hyper9 and InfoWorld in his article, “VM stall: Breaking through the second phase of virtualization“:

“… many organizations strike a ‘perfect storm’ of challenges that slows their virtualization rollout, or stops it entirely. Some causes at this stage include greater complexity of services and applications, higher demand on scarce virtualization skills, limited visibility into a growing deployment, increasingly heterogeneous systems, and greater resistance from risk-averse application owners and recalcitrant application vendors.”

In the same article, David spoke with Dave Bartoletti, formerly of automation vendor Enigmatec and now a leading light showing the way through the virtualization darkness with research and advisory analyst firm, the Taneja Group:

“The second wave of issues is always harder when a core technology matures. Server virtualization essentially paid for itself in CAPEX savings, but when we virtualize Tier 1 business-critical applications, or user desktops, CAPEX savings take a backseat to application performance and IT efficiency, and this is why we’re stalling.”

My former editor at Tech Target and another keen virtualization observer, Colin Steele, highlighted another core element of VM stall, in his article “ISV stall makes virtualizing applications a challenge“:

By now, the benefits of virtualizing applications are clear, but the goal of 100% virtualization remains elusive. One reason is that some independent software vendors (ISVs) don’t support their server-based applications — databases, telecom apps, healthcare programs, etc. — on virtual servers.

Moreover, I talk a lot with customers about their real world concerns, so I can quickly pinpoint many other causes. They talk to me about issues like vendor licensing, facilities constraints, capacity blindness, service prioritization, deployment costs, line-of-business resistance, internal politics, a lack of skills, and even senior management resistance.

In fact, last week at CA Expo in Australia, I talked with CA Technologies customers about seven significant issues in virtualization that are contributing to (among other things) VM stall, as you can see from one of the slides from my presentation:

Virtualization is not clear sailing - from CA Expo Australia

(You can see the whole deck at the CA Expo site)

To be fair to Jay and his team, other posts on his corporate blog agree with me, citing issues like mission-critical apps, management skepticism, bureaucracy, poor project vetting, and more.

I am really glad to see my thoughts around VM stall have captured the imagination of the market. Thanks to Jay for taking this up, and to his team for joining me and CA Technologies in raising awareness of issues causing VM stall.

However, I think we all need to be careful about being categorical about VM stall. It is important to be clear that VM stall – like most enterprise IT issues, and indeed most organizations – is both complex and varied, so trying to categorically define four (or six, or seven, or really any number) of causes for VM stall is underestimating this important problem.

But if we can all contribute new ideas to the community, we will all learn more, and our enterprise customers will benefit from our combined wisdom.

Most devops discussions are missing the target

I am hearing a lot about the rise of a concept called ‘devops’ – a mashup of ‘development’ and ‘operations’. I am not at all an expert in this area, but from what I can tell, devops is aimed at streamlining rapidly iterative application delivery to allow for greater development and business agility. Devops aims to achieve this by breaking down the barriers – human, process, and technology – between application development and system operations.

Interestingly, the concept is new enough that, as I write this, there is not even an entry for it in Wikipedia yet. I did find a blog by Damon Edwards (on Twitter – @damonedwards) very useful though, as he explains the age-old disconnects between application developers ‘throwing software over the wall’, and ops who are painfully resistant to change. James Urquhart (@jamesurquhart ) blogged very recently on the concept too , and again provided some very helpful content. Conversing online with them and others also helped me to formulate some more concrete ideas about devops – or at least some more concrete questions.

My interest was especially piqued when I understood how closely devops is connected to virtualization, cloud, and automation – my core interests:

• Cloud – Devops has antecedents in ‘rogue’ developers (or developers from smaller shops) using cloud resources (IaaS, PaaS) for new projects, and will benefit greatly from cloud-based development and deployment, as cloud providers do not impose the restrictions of internal change-averse ops teams, and developers can essentially manage their own ops requirements instead.
• Virtualization – In-house devops (which needs more heavy lifting) is greatly assisted by virtualization, as virtual machines become the new base unit for application packaging, avoiding application rollout failures caused by incompatibility between the test and production environments (hardware, OS, middleware, etc.).
• Automation – In-house devops is also greatly facilitated by automation, which can use standard workflows to automatically provision and configure these complete application VMs, as well as backup and restore VMs, allowing complex composite application deployment and rollback at the click of a mouse.

Clearly devops has many very attractive outcomes – drive agile business, reduce delays, smooth application releases, deliver value faster. It is a very seductive idea. Who wouldn’t want it?

However, most of the writings I see about devops are really about dev, not ops. As a result, they don’t really capture the whole story of the application lifecycle. They justify devops as an antidote to the problems that ops are causing – slowing down release cycles, imposing arbitrary rules, screwing up deployments, killing developer productivity, hacking manual scripts and configs, stopping the business from being agile – but fail to recognize both the failings of developers that contribute to the problems, and the role of operations in delivering critical business outcomes during the application delivery lifecycle.

On the contrary, discussions mainly focus on how developers can sideline or change operations, positioning devops as the lone hero in the battle against inefficiency, as application developers fix all the problems (!) by controlling or automating key release management operations like provisioning, deployment, integration, patching, and software update. Meanwhile, ops are marginalised, along with their timesinks and roadblocks, satisfying the needs of an agile and rapidly changing business.

See – seductive, isn’t it?

Yet this seems to me (as a former op) fundamentally flawed, a development-centric neologism based on an incomplete understanding of the real purpose and role of IT operations, or of operations’ history in the development of ‘agile’ IT.

The way I see it, devops misses that target on how IT ops serve business needs too, and seems to gloss over ‘coal face’ realities like:

• Who handles ongoing support, especially software update for the unrestrained sprawl of non-standard systems and components.
• Who ensures each new application doesn’t interfere with existing and especially legacy systems (and networks, storage, etc.)?
• Who handles integration with common production systems that cannot be encapsulated in a VM, like storage arrays (NAS, SAN), networking fabrics, facilities, etc.
• Who handles impact analysis, change control and rollback planning to ensure deployment risk is understood and mitigated?
• Who is responsible for cost containment and asset rationalization, when devops keeps rolling out new systems and applications?
• Who ensures reporting, compliance, data updates, log maintenance, Db administration, etc. are built into the applications, and integrated with standard management tools?
• Who will assure functional isolation, role-based access controls, change auditing, event management, and configuration control to secure applications, data, and compliance?

Because, if you have ever worked with both ops and apps, you know it is not going to be apps.

Now, in defence of devops, I am sure it is being implemented and conferring major benefits, especially in small organizations with little IT management discipline. I am sure the supporters of devops have some positive goals in mind too. What’s more, it is addressing a very real problem – ops really should spend more time on better processes and controls than in ‘daily deployment muck’.

However, devops should be a two-way street. As a former op, I know that the apps team have to pull their weight too, by addressing gaps like:

• Including ops during the design process, so applications are built to work with standard ops tools
• Taking ops input on deployment, so applications will go in cleanly without disrupting other users
• Working with ops on capacity and scalability requirements, so they can keep supporting it when it grows
• Implementing ops’ critical needs for logging, isolation, identity management, configuration needs, and secure interfaces so the app can be secure and compliant
• Giving ops some advance insight into applications, especially during test and QA, so they can start to prepare for them before they come over the wall
• Allowing ops to contribute to better application design, deployment, and management; that ops can do more for the release cycle and ongoing management than just ‘manipulating APIs’

See, ops do enable business – and agile business at that – by ensuring that new applications coming into an existing complex environment are safe, secure, reliable, integrated, and responsive, regardless of how complex IT is, or how many moving parts there are. Devops seems to miss this important detail.

So I am sceptical of how devops will work in large, well-run IT environments with important and necessary operational controls, especially the > 60% of organizations that are committed to ITIL best practices (like formal and integrated management of change, configuration, release, assets, etc.).

After all, ‘agile’ does not magically obviate the need to identify and prevent bad changes, to reject apps that breach operational compliance, to ensure each new application adheres to standards, or to prevent uncontrolled sprawl of heterogeneous software.

I still have a lot to think about on this topic, and am trying to keep an open mind. But my best guess right now is that, for enterprises at least, devops either will not take hold or will not last. It seems most likely to be instead, at best, a transitory state on the path to a ‘new normal’. As with all ‘revolutions’, it has started outside IT ops, yet I expect will eventually co-opt and migrate wholly to operations in some form. Once the revolutionaries in development understand how many business needs besides agility actually require  routine, process, management, and controls, they will back away from devops the same way they backed away from ownership in other IT revolutions – like the deployment of mini computers, desktops, and web applications.

If it does turn out this way – don’t worry. Operations will again dutifully take the reins, and clean up the mess that devops will leave behind. Because that is what ops do – they manage what they are given, and keep the business running, regardless of the mess that gets thrown over the wall at them.

In any case, whether devops takes root or not, hopefully we will all learn something about cooperation, automation, agility, and control. Because all stakeholders in the devops discussion – development, operations, and business owners – could benefit from that.

The only significant and unique benefit of KVM for server virtualization (as noted by Sander van Vugt in our (virtual) debate on Xen vs.KVM Linux Virtualization Hypervisors) is that KVM is part of the Linux kernel. This ensures broad standardization, patch compatibility, simpler upgrades, and a low-impact on-ramp for existing Linux IT shops.

Yet this is a solution for a problem that does not really exist.

Large enterprises already run thousands of components, from services/daemons to drivers to applications, all as additions to various kernels. Maintaining one more (or even several more) non-kernel components like Hyper-V, XenServer, ESX, etc., is not a net negative. On the contrary, EMA data shows that virtualization actually improves the productivity of server administrators, and by an average of around 10% – up to 20% or more for best performers. For competent administrators with good lifecycle management tools, the time they spend to learn, test, and maintain hypervisors is a significant effort, but it is time paid back with interest.

On the other hand, many downsides to KVM are all too apparent.

It is easy to point to the lack of technology features and maturity in KVM – areas like live migration, paravirtualization, networking, isolation, performance, security, or a host of other  features which KVM (in some cases arguably) lacks. I have only some doubt that KVM will meet these low-level functional requirements eventually, but it will not be anytime soon. Yet they are essentially table stakes in server virtualization today.

The inherent dependency on Linux would also require a major shift in  platforms for the average datacenter (where Windows outnumbers Linux by  150:1), and a major investment in resourcing, training, and software. This is hardly an attractive proposition for a data center manager. Still, existing Linux staff will be able to pick it up, and could even have some success on their (relatively few) existing Linux platforms.

However, even if these weaknesses are overcome, KVM has a much more strategic problem – the gaping void in the KVM management ecosystem. There is almost no third-party support for KVM from management vendors. Even stated support from key partner vendors like IBM, HP, and of course Red Hat is basic at best. What’s more, EMA data suggests KVM will not foster a significant management ecosystem in the future, either.

EMA’s research on Virtual System Management showed convincingly how important management is to virtualization. Across 18 different management disciplines, almost all correlated with measurably better outcomes in metrics like MTTR, provisioning time, availability, VM density, migration speed, and more.

EMA’s new cloud research shows a similar importance. Applying mature automation and management disciplines to virtual systems is directly correlated with positive cloud outcomes like reduced CapEx, reduced OpEx, improved operational maturity and more.

Not surprising then, that over 80% of enterprises consider manageability an important or very important factor in their virtualization and cloud technology decisions.

Unfortunately, KVM ranks anywhere from 4^th to 10^th in enterprise preferences for virtualization and cloud technology providers. It comes behind first ESX, then Hyper-V or Xen (multiple implementations), often various UNIX hypervisors (PowerVM, Integrity VMs or vPars, Solaris Containers), and even z/VM. No enterprise demand means that management vendors have little incentive to support KVM.

In fact, in my conversations with management software vendors, most generally put KVM around 5th in line for support – which, realistically, means it is not even on the current roadmap. What’s more, for better or worse several of them have a vested interest in not supporting KVM (no points for guessing who).

This means KVM has little or no prospect of gaining third-party support for virtualization management tools like VM-aware backup and restore, VM provisioning, virtual resource management, VM configuration auditing, virtual performance monitoring, VM lab management, VM image control, storage management,network automation and more. The same holds true for integration with higher-level virtual systems management tools for virtual and physical data center automation and service management disciplines.

For any IT group, sophisticated management tools deliver many proven benefits. For larger enterprises especially, they are simply not optional.  Without even the prospect of a robust management ecosystem, KVM is simply a non-starter in most large-scale deployments. For my enterprise clients at least, it is certainly not a credible choice for x86 server virtualization.

Enter Intelligent Workload Management (IWM), which, according to the Novell press release is:

… Novell’s differentiated approach to Intelligent Workload Management [that] integrates identity and systems management capabilities into an application workload, thereby increasing the workload’s security and portability across physical, virtual and cloud environments

All I can say is … bravo Novell!

No, really. It is about time. Novell has exceptional capabilities in virtualization, automation, and service management; and it also adds critical capabilities for security management and compliance, especially around identity management.  These are all core values in what EMA calls ‘the responsible cloud’.

The EMA thesis, essentially, is that cloud computing has too many cowboys, and not enough sheriffs. Enter Novell, the “Doc” Holliday of the cloud landscape, with responsible capabilities for virtualization, automation, service management, and security and compliance.

IBM, Microsoft, Sun, and even Oracle might argue with Novell in some of its claims of uniqueness – after all, all of them have substantial capabilities in all these areas too.

However, regardless of some overreaching in their marketing, competitive threats, a nascent market, and gaps in actual product capability, Novell has an excellent opportunity to re-brand itself and deliver some exceptional capabilities to deliver on private cloud computing goals, and is as well positioned as any vendor to stake a claim to what they label ‘Intelligent Workload Management’.

Keep an eye out for EMA’s more detailed Impact Brief on this announcement. Very interesting stuff, without doubt.

Andi.

Good for them!

What I liked most was that they are trying to break down the barriers between systems and security management. Certainly this is something that I discuss regularly with enterprises – the need to stop focusing on silo-based management, and perhaps even more importantly, to stop pandering to silo-focused low-level managers. Almost all of the CIOs, VPs, and IT Directors who I talk with are critically aware of the problems these silos cause – including human errors, resource inefficiencies, security problems, and higher costs.

This is also a constant discussion I have within EMA, especially with the lead of our security practice, Scott Crawford – a brilliant mind on security (amongst many other subjects) who constantly thinks about security in ways I never could or would. We work and publish together on this topic frequently. Indeed, it has come up again in our latest research, which shows that security and risk management are a fundamental requirement for cloud computing – or what EMA calls the ‘Responsible Cloud’.

The upshot of all these conversations is simple – security management and systems management are not, cannot, and should not be completely separate. Not in human terms, not in processes, and not in technologies. Without doubt, anyone in a large enterprise who has ever tried to implement a patch, a configuration change, a firewall update, a software release, or a hundred other data center changes will attest to this in a heartbeat.

Of course (as Scott rightly pointed out when I last spoke with him about this), we will always need security experts, and systems experts – the two disciplines are not the same, and we will always need deep domain expertise in each. So I am not advocating complete convergence. But we need more software tools that provide integration and interoperability that allow these professional to work more effectively together.

While multi-function vendors like CA, Symantec, IBM, and others have the product portfolio to approach these cross-silo problems holistically, there are few ‘best of breed’ vendors thinking this way. Of course, Tripwire and the ever-inspiring Gene Kim (who I have sadly never met) spring to mind for me; so would Configuresoft (although now as part of EMC Ionix, hardly a niche vendor), and the indefatigable Dennis Moreau. Both inspire their teams, technologies, and customers by championing a fundamental understanding that systems and security cannot, at their heart, be completely separated.

(As an aside, these two seem like they would have been incredibly compelling arch-enemies in some ubergeek superhero genre – although I would never want to choose which should be the hero and which the villain!)

I must say that, so far at least, I don’t know the product design team from Reflex personally – guys like Hezi Moore, Aaron Bawcom, and Mike Wronski – as well as I do Dennis or Gene. However, I do know that they all have very credible security chops. Plus, one thing is clear to me.

They get it. They really get it.

And that in itself is a thing of rare beauty.

Deliberately designing functionality that addresses both security and systems management – like functional isolation, integrated access control, change segregation, granular audit trails, policy based management, and role-based access – into a systems management toolset is a rare feat, especially in startup and niche products. It is something I look for all the time, because my enterprise clients often demand it. Sadly, all too often I fail to find it – and I am not even a real security wonk! When I do, I am pleasantly surprised. When I see deep thought going into the security value of a systems management product, I am almost ecstatic.

Unfortunately, the challenge for vendors like Reflex and Tripwire (as it was for Configuresoft, and perhaps is still for EMC, Symantec, etc.) is to find customers that value this synergy. While most high-level IT execs understand this imperative, their holistic view frequently does not translate to many of their lower-level managers, or to many functional IT practitioners.

Of course, there are plenty of departmental ops managers and security managers who do get it. They strive to connect their teams with other groups, driving greater business efficiency and effectiveness as a result. However, unfortunately, many do not, instead focusing on protecting their small empires, walling themselves off from integrated management and cross-functional resourcing.

Similarly, many positive-minded individual technicians will actively seek out cross-skilling opportunities, recognizing that it makes them not just more useful but also more valuable, and more indispensable. However, many practitioners (both security and ops) can be just as bad as the most myopic managers (who they often work for), dogmatically eschewing integrated management tools and processes, seeing them as threats to their own personal domains of control.

Sad but true, best practices like breaking down IT management silos are not always adopted.

Fortunately, vendors like Reflex and Tripwire that have expertise and passion in both ops and security (and – shameless plug – trusted advisors like EMA, which is big enough to have experts in both disciplines, yet small enough that we still work together), are trying to break down these barriers.

And more power to them. They serve their clients much better by promoting the undeniable facts that security values are critical to systems management, and systems management is critical to security.

The survey revealed some very interesting data, with a very well thought out instrument and a quality sample – 300 respondents (100 each from the US, EMEA, and Asia Pacific) with at least 500 employees in the US (250 in the UK, France, Germany, Australia, Singapore and India), and all with a current or planned investment in server virtualization.

A number of data points stand out for me:

• The balance of Test/Dev implementations vs. Production continues to reflect EMA data. While production is still lagging behind test and dev as a use case, virtualization for mission-critical production is only slightly behind test and dev overall – and within the margin of error in most cases. This is good news, as enterprises clearly continue to grow real, production use cases.  It was interesting to see the differences between US and EMEA/APAC on this data point too, something EMA has not broken out in our published reports.
• Microsoft and VMware are neck and neck in enterprises’ plans for server virtualization deployments over the next 18 months. I was called crazy when my 2008 EMA research pointed to a 32% growth rate for Microsoft Hyper-V into 2009, trailing only VMware; yet here we are in 2009, and according to this new study, through 2010/11 that is going up to 49%. So who is crazy now?
• The strong growth for endpoint (desktop, application) virtualization reflects EMA data very well. It also highlights where enterprises and vendors should be heading with management technologies. It is still early days, but there are  a lot of gaps in integrated management for physical and virtual endpoints. So it is clear that this (probably even more than cloud service management) is going to be the next big problem for IT management.
• The percentage of IT services planned to be virtualized over 18 months is growing well. However, just as EMA has predicted, virtualization will remain at only around 50% of service deployments even through 2011, so there will continue to be substantial physical deployments. This reinforces my consistent (and insistent) position that effective management of virtualization must integrate both physical and virtual systems management
• Effective management continues to be elusive. EMA’s research showed this in 2006, 2008, and 2009, and this new data (with some reservations) shows the same. However, while tThe majority (64%) of enterprises rank themselves as extremely or very effective at managing virtualization, and believe they are getting better, I am skeptical. I contend many of those are overestimating their abilities (see my next points).
• Virtualization clearly increases complexity, and is clearly more difficult to manage. I felt like I was tilting at windmills when I published this opinion in 2006 and in 2008, contrary to common perceptions that virtualization made everything easier. I was certainly a lone voice, but as it turned out, a prescient one. It is great to see it being recognized more broadly, finally.
• Human issues continue to be major problems – especially skills and resourcing. EMA has found the same to be true, consistently, for many years. As recently as last week, I spoke with David Marshall of VMBlog and InfoWorld about how this continues to be a problem. This makes management tools even more important – to embed knowledge, define and execute policy, and automate routine work to free up resources.
• These data points all increase my doubt that enterprises are really being better at managing their virtual environments. It seems contradictory to me that this survey shows virtualization is more complex, management is the top inhibitor to ROI, and skills are still lacking, yet most enterprises think they are being very or extremely effective at it. Even though tool usage is more integrated and automated than it has been, this does not make sense. I am instead convinced that enterprises are really overestimating their abilities.

This is just a very small sample of the interesting data in this survey – there are more than 50 pages in the slide deck I reviewed ahead of the webcast. I encourage you to check out the webcast.  If you get in touch with HP, I am sure they will point you in the right direction; or check back here, and I will post the link when it is up.

Andi.

In part 2 of podcast, Carl and I talk about:

• cost details, pricing, and licensing models for cloud management tools
• options for open source and ‘freemium’ tools for cloud computing management
• the problems of an old-school approach to cloud computing management
• the issues of paying for cloud computing management using traditional models
• the critical dangers associated with a slow adoption of cloud computing management
• the need to integrate with existing SOA, reporting, and other in-house disciplines
• difficulties of managing on-premise, off-premise, and hybrid clouds
• the real long-term ROI and NPV of cloud computing and management
• the importance of analytics, chargeback, capacity management, and accountability in predictable cost models for cloud computing
• the difficulty of visibility, verifiability, predictability, and the trust relationship with cloud computing providers
• and what to look for in both pricing and general comfort levels

It was a great session. You should check it out at TechTarget’s SearchCloudComputing.com site!

Andi.

It is described on TechTarget as follows:

While not distinctly different from traditional data center management, cloud computing management presents a new environment and new situations for vendors and users to master in order to keep a watchful eye on their applications and data.

Learn about the meaning of cloud management, its growing purpose in the cloud computing world and the expectations laid on cloud management providers in this podcast with Andi Mann, vice president of research with Enterprise Management Associates.

In the podcast, Carl and I talk about:

• specific disciplines in cloud management like threat management, performance management, availability, malware detection, provisioning, and automation
• the offerings and suitability of different vendors like Hyperic (now part of VMware), Rightscale, CA, BMC Software, and Nimsoft
• options for small and medium business (SMB) vs. large enterprises
• benefits and issues with home-grown, do-it-yourself (DIY), or scripting approaches vs. packaged software
• cloud computing and cloud management vendor lock-in
• who needs cloud service management
• the need for threat and vulnerability management (and impact of not having it)
• who needs to deploy cloud computing management tools
• and when to consider management tools for cloud computing

Check out the full podcast at TechTarget’s SearchCloudComputing.com site!

Part 2 will be coming soon too. I will letyou know when it is up.

Andi.

For example:

• Virtual appliances add an unknown operating system to the environment. It is typically a slimmed-down Linux distro, but you rarely know – it could be DR-DOS 6.2 or a pirate copy of Windows ME. This breaks any software SOE, ignoring top level decisions on OS stability, reliability, longevity, security, etc.
• Administrators have virtually no control over virtual appliance management. Management functions are required for any software, but virtual appliances rely entirely on a middle-man for proper OS, middleware, application, and database patches & upgrades, malware detection, performance monitoring, problem analysis, etc.
• Even when ad hoc management is possible, it is almost always manual. You can’t put agents on most virtual appliances, they don’t come with WMI, and most have only a GUI for management. So you cannot use standard tools or automation, which wastes admins’ time, risks audit non-compliance, and invites human error.
• Security is a particular concern. Timeliness of patches, effectiveness of hardening processes, zero-day threat response, malware protection, and so on are all at the whim of the vendor, and rarely disclosed to the customer.
• You pretty much have to pay maintenance. If you don’t, chances are you simply cannot keep a virtual appliance up-to-date yourself.

Of course, many of the same criticisms can be slated against physical appliances. I have even talked with one enterprise that will not deploy even physical management appliances because they would break the company’s hardware SOE (even though network devices, storage systems, and other ‘boxes’ are often just purpose-built appliances). However, with just an Ethernet cable connecting them to the enterprise, and a generally slimmer system profile, they seem to pose a lesser risk. They are also much simpler than virtual appliances, which add (in many cases unnecessarily) a layer of complexity and abstraction that physical appliances do not, by virtue of being encapsulated within a virtual machine. Moreover, the resources and effort to build a ‘real’ appliance is far greater than just slapping some software into a virtual machine, so physical appliance vendors seem somehow more committed, more reliable.

Is this distinction fair? Possibly not. But regardless of my own concerns, my research has shown that virtual appliances are the least-preferred of any form factor for management software, with physical appliances, niche software, and even software suites more preferred. Really, when the dreaded ‘framework’ is more popular than you, perhaps you really are an ugly duckling.

Which is not to say that virtual appliances are pointless. They are easy to implement, provide fast time-to-value, and are especially good for trials and POCs. They require little or no tuning, and the OS environment is often a bare bones install which is fast and efficient. Unlike physical appliances, they are easily scalable, and highly mobile. They can be deployed in seconds (maybe minutes) even to far-flung locations in regional offices with zero travel time and cost. And they allow even a sysop to deploy a new management server without getting the network, storage, security, or server teams involved. All of these are powerful factors in their favour.

I am also seeing, despite their potential issues, that several vendors are being very successful selling virtual appliances. KACE, for example, told me today that 26% of their total sales in Q3′09 have been of their virtual appliance, the V-KBOX; VKernel provide all their software in virtual appliance formats, and their Q3′09 sales were 205% up on Q3′08; Citrix is finding a remarkable early demand for their Netscaler VPX virtual appliance.  Meanwhile, IBM, Symantec, up.time, Reflex, SourceFire, and several others are agressively in or entering the market for management systems delivered as virtual appliances.

I also think that virtual appliances have a bright future – but in some ways I continue to see them as a beta version of what could (or should) come next.  By adding in capabilities for responsible and accountable management, they could form the basis of more fully-functional virtual service management containers. These in turn could form the basis of elastic, mobile, network-deployed, responsible cloud appliances that deliver complete end-to-end service management without regard to physical location or domain of control.

A couple of vendors are clearly headed this way, but even without this level of sophistication and maturity,  it certainly seems like vendors and buyers are increasingly embracing virtual appliances, despite their many potential flaws.

Perhaps I should too?

Andi.