More than four reasons count towards VM stall

I recently saw a great article in IT World Canada titled “Virtual stall: What it is and why you have it,” written by Jay Litkey, that took up my idea of VM stall, which I first came up with in my blog from May ‘Is “VM Stall” the Next Big Virtualization Challenge?‘.

Though they barely acknowledge my blog as their inspiration (and as a competitor to CA Technologies – my employer – why would they?), it seems Jay and his team have wholeheartedly taken up my concern with VM stall, and not just in the IT World Canada article. Marketing lead David Lynch was quoted on the topic in a post by Bruce Hoard of Virtualization Review, and in a recent Tech Target article on ‘ISV stall’. Several posts on their corporate blog also address the issue as if it was their own baby.

In my past life at EMA, I have spoken with both Jay and David a number of times, and had a lot of time for what they were doing in the management space. For a small startup with limited resources, it is great that they can take the time to pick up my idea and run with it.

The IT World Canada article is really worthwhile, because it zeroes in on some important concepts. It helps to expand the thought around VM stall, and specifically on a couple of additional causes, as it notes:

Virtual stall has four main causes:

• Scalability issues:  A single IT team often finds it difficult to scale beyond the 25-30 per cent penetration range. This is due to the combination of lack of automation and reporting in virtualization management tools, creating time-consuming manual processes that are a particular problem when there is a lack of experienced and trained staff.
• Management issues: The data centre is not a place that can be managed manually; there are too many elements to be checked, and too many independencies [sic]. And, while there are levels of automation built into the virtualization platform, they can be difficult to define and implement. The lack of automated monitoring, alerting and control becomes more and more of a problem as the overall level of virtualization in the data centre increases.
• Process issues:  Enterprise virtualization impacts a wide range of existing data centre processes, all of which need to be modified, replaced, or augmented. As long as the virtual environments are small and self-contained, these processes can be manipulated or ignored. But as the environment grows, it reaches a point when they have to be dealt with before real efficiencies can be reached. The more “process-mature” an organization is, the more quickly this point is reached.
• Co-ordination issues: Virtualization crosses multiple silos and ultimately requires a level of co-operation and integration that is impossible to achieve with the traditional silo management structure. In addition, the first workloads to be virtualized tend to be less critical ones.  However, as environments grow, higher-risk, higher-impact services are virtualized. These tend to have more stakeholders, more politics, more distributed infrastructures, and a greater cost of failure and downtime. Consequently, they require more coordination.

This is great insight, and offers a number of important causes. However, I don’t think it is reasonable to say there are just “four main causes.” Not to pick on Jay, as it is probably just unfortunate phrasing, but I think it is important to see that the issues of VM stall are much more varied, complex, and numerous.

I am not entirely without fault either. To start with, when I first identified the issue of VM stall in my blog post back in May, I said that “I see many possible causes for VM stall,” but like Jay I only identified four examples. As Jay recounts in his analysis, I saw scalability and manageability as key issues; but unlike Jay, I chose to highlight risk aversion and resourcing as two more of my examples.

However, even these six are just a part of the problem. As I said when I spoke with my great mate (and one of the industry’s great virtualization gurus, observers, and commentators), David Marshall of Hyper9 and InfoWorld in his article, “VM stall: Breaking through the second phase of virtualization“:

“… many organizations strike a ‘perfect storm’ of challenges that slows their virtualization rollout, or stops it entirely. Some causes at this stage include greater complexity of services and applications, higher demand on scarce virtualization skills, limited visibility into a growing deployment, increasingly heterogeneous systems, and greater resistance from risk-averse application owners and recalcitrant application vendors.”

In the same article, David spoke with Dave Bartoletti, formerly of automation vendor Enigmatec and now a leading light showing the way through the virtualization darkness with research and advisory analyst firm, the Taneja Group:

“The second wave of issues is always harder when a core technology matures. Server virtualization essentially paid for itself in CAPEX savings, but when we virtualize Tier 1 business-critical applications, or user desktops, CAPEX savings take a backseat to application performance and IT efficiency, and this is why we’re stalling.”

My former editor at Tech Target and another keen virtualization observer, Colin Steele, highlighted another core element of VM stall, in his article “ISV stall makes virtualizing applications a challenge“:

By now, the benefits of virtualizing applications are clear, but the goal of 100% virtualization remains elusive. One reason is that some independent software vendors (ISVs) don’t support their server-based applications — databases, telecom apps, healthcare programs, etc. — on virtual servers.

Moreover, I talk a lot with customers about their real world concerns, so I can quickly pinpoint many other causes. They talk to me about issues like vendor licensing, facilities constraints, capacity blindness, service prioritization, deployment costs, line-of-business resistance, internal politics, a lack of skills, and even senior management resistance.

In fact, last week at CA Expo in Australia, I talked with CA Technologies customers about seven significant issues in virtualization that are contributing to (among other things) VM stall, as you can see from one of the slides from my presentation:

Virtualization is not clear sailing - from CA Expo Australia

(You can see the whole deck at the CA Expo site)

To be fair to Jay and his team, other posts on his corporate blog agree with me, citing issues like mission-critical apps, management skepticism, bureaucracy, poor project vetting, and more.

I am really glad to see my thoughts around VM stall have captured the imagination of the market. Thanks to Jay for taking this up, and to his team for joining me and CA Technologies in raising awareness of issues causing VM stall.

However, I think we all need to be careful about being categorical about VM stall. It is important to be clear that VM stall – like most enterprise IT issues, and indeed most organizations – is both complex and varied, so trying to categorically define four (or six, or seven, or really any number) of causes for VM stall is underestimating this important problem.

But if we can all contribute new ideas to the community, we will all learn more, and our enterprise customers will benefit from our combined wisdom.

Most devops discussions are missing the target

I am hearing a lot about the rise of a concept called ‘devops’ – a mashup of ‘development’ and ‘operations’. I am not at all an expert in this area, but from what I can tell, devops is aimed at streamlining rapidly iterative application delivery to allow for greater development and business agility. Devops aims to achieve this by breaking down the barriers – human, process, and technology – between application development and system operations.

Interestingly, the concept is new enough that, as I write this, there is not even an entry for it in Wikipedia yet. I did find a blog by Damon Edwards (on Twitter – @damonedwards) very useful though, as he explains the age-old disconnects between application developers ‘throwing software over the wall’, and ops who are painfully resistant to change. James Urquhart (@jamesurquhart ) blogged very recently on the concept too , and again provided some very helpful content. Conversing online with them and others also helped me to formulate some more concrete ideas about devops – or at least some more concrete questions.

My interest was especially piqued when I understood how closely devops is connected to virtualization, cloud, and automation – my core interests:

• Cloud – Devops has antecedents in ‘rogue’ developers (or developers from smaller shops) using cloud resources (IaaS, PaaS) for new projects, and will benefit greatly from cloud-based development and deployment, as cloud providers do not impose the restrictions of internal change-averse ops teams, and developers can essentially manage their own ops requirements instead.
• Virtualization – In-house devops (which needs more heavy lifting) is greatly assisted by virtualization, as virtual machines become the new base unit for application packaging, avoiding application rollout failures caused by incompatibility between the test and production environments (hardware, OS, middleware, etc.).
• Automation – In-house devops is also greatly facilitated by automation, which can use standard workflows to automatically provision and configure these complete application VMs, as well as backup and restore VMs, allowing complex composite application deployment and rollback at the click of a mouse.

Clearly devops has many very attractive outcomes – drive agile business, reduce delays, smooth application releases, deliver value faster. It is a very seductive idea. Who wouldn’t want it?

However, most of the writings I see about devops are really about dev, not ops. As a result, they don’t really capture the whole story of the application lifecycle. They justify devops as an antidote to the problems that ops are causing – slowing down release cycles, imposing arbitrary rules, screwing up deployments, killing developer productivity, hacking manual scripts and configs, stopping the business from being agile – but fail to recognize both the failings of developers that contribute to the problems, and the role of operations in delivering critical business outcomes during the application delivery lifecycle.

On the contrary, discussions mainly focus on how developers can sideline or change operations, positioning devops as the lone hero in the battle against inefficiency, as application developers fix all the problems (!) by controlling or automating key release management operations like provisioning, deployment, integration, patching, and software update. Meanwhile, ops are marginalised, along with their timesinks and roadblocks, satisfying the needs of an agile and rapidly changing business.

See – seductive, isn’t it?

Yet this seems to me (as a former op) fundamentally flawed, a development-centric neologism based on an incomplete understanding of the real purpose and role of IT operations, or of operations’ history in the development of ‘agile’ IT.

The way I see it, devops misses that target on how IT ops serve business needs too, and seems to gloss over ‘coal face’ realities like:

• Who handles ongoing support, especially software update for the unrestrained sprawl of non-standard systems and components.
• Who ensures each new application doesn’t interfere with existing and especially legacy systems (and networks, storage, etc.)?
• Who handles integration with common production systems that cannot be encapsulated in a VM, like storage arrays (NAS, SAN), networking fabrics, facilities, etc.
• Who handles impact analysis, change control and rollback planning to ensure deployment risk is understood and mitigated?
• Who is responsible for cost containment and asset rationalization, when devops keeps rolling out new systems and applications?
• Who ensures reporting, compliance, data updates, log maintenance, Db administration, etc. are built into the applications, and integrated with standard management tools?
• Who will assure functional isolation, role-based access controls, change auditing, event management, and configuration control to secure applications, data, and compliance?

Because, if you have ever worked with both ops and apps, you know it is not going to be apps.

Now, in defence of devops, I am sure it is being implemented and conferring major benefits, especially in small organizations with little IT management discipline. I am sure the supporters of devops have some positive goals in mind too. What’s more, it is addressing a very real problem – ops really should spend more time on better processes and controls than in ‘daily deployment muck’.

However, devops should be a two-way street. As a former op, I know that the apps team have to pull their weight too, by addressing gaps like:

• Including ops during the design process, so applications are built to work with standard ops tools
• Taking ops input on deployment, so applications will go in cleanly without disrupting other users
• Working with ops on capacity and scalability requirements, so they can keep supporting it when it grows
• Implementing ops’ critical needs for logging, isolation, identity management, configuration needs, and secure interfaces so the app can be secure and compliant
• Giving ops some advance insight into applications, especially during test and QA, so they can start to prepare for them before they come over the wall
• Allowing ops to contribute to better application design, deployment, and management; that ops can do more for the release cycle and ongoing management than just ‘manipulating APIs’

See, ops do enable business – and agile business at that – by ensuring that new applications coming into an existing complex environment are safe, secure, reliable, integrated, and responsive, regardless of how complex IT is, or how many moving parts there are. Devops seems to miss this important detail.

So I am sceptical of how devops will work in large, well-run IT environments with important and necessary operational controls, especially the > 60% of organizations that are committed to ITIL best practices (like formal and integrated management of change, configuration, release, assets, etc.).

After all, ‘agile’ does not magically obviate the need to identify and prevent bad changes, to reject apps that breach operational compliance, to ensure each new application adheres to standards, or to prevent uncontrolled sprawl of heterogeneous software.

I still have a lot to think about on this topic, and am trying to keep an open mind. But my best guess right now is that, for enterprises at least, devops either will not take hold or will not last. It seems most likely to be instead, at best, a transitory state on the path to a ‘new normal’. As with all ‘revolutions’, it has started outside IT ops, yet I expect will eventually co-opt and migrate wholly to operations in some form. Once the revolutionaries in development understand how many business needs besides agility actually require  routine, process, management, and controls, they will back away from devops the same way they backed away from ownership in other IT revolutions – like the deployment of mini computers, desktops, and web applications.

If it does turn out this way – don’t worry. Operations will again dutifully take the reins, and clean up the mess that devops will leave behind. Because that is what ops do – they manage what they are given, and keep the business running, regardless of the mess that gets thrown over the wall at them.

In any case, whether devops takes root or not, hopefully we will all learn something about cooperation, automation, agility, and control. Because all stakeholders in the devops discussion – development, operations, and business owners – could benefit from that.

Effort vs. Payback is an Everyday Business IT Decision

I read recently a good blog post from Thomas Bittman (@tombitt) of Gartner Group, about how sometimes close enough is good enough. Talking specifically about private cloud, he talked about how an ‘imperfect’ cloud deployment – one that does not have all five essential characteristics, for example – might be enough for some organizations.

I especially appreciated how he highlighted some very specific, real-world examples to sustain his advice. As he shows, sometimes you don’t need a ‘100%’ implementation, and for very good business reasons.

Not every IT organization needs a fully self-service interface, and many smaller organizations see no value in usage metering. They simply want to deliver services faster. For them, a 70% private cloud is absolutely good enough … it all comes down to business requirements, return on investment, and future strategy. How far you go is your decision.

via Driving for Imperfection With Your Private Cloud.

If you haven’t seen it yet, you should. It’s a quick read, only 4 paragraphs and less than 300 words. Go ahead. I’ll still be here when you get back.

The theme is very similar to something I wrote in a research report for EMA, ‘The Responsible Cloud‘, also on cloud computing. Regarding the NIST definition of cloud, I cautioned against dogmatic interpretations of cloud computing, and the notion that a ‘real’ cloud must necessarily have all of the essential characteristics, or fit some specific deployment model. Flexibility is key, I advised, and delivering on key business requirements is more important than definitions.

Two other things happened this week that made me think about this in different ways:

• An internal session at CA reviewing some customer-facing materials. All attendees agreed – we can’t preach unattainable dogma; we need to deal with specific requirements and partial deployments, as well as broad requirements that come from  ‘100%’ implementations.
• A group discussion on LinkedIn, where an IT practitioner wanted advice on building a small private cloud. He was soon inundated with an unrealistic list of requirements, from hypervisor features to management disciplines, that he *must* have to build a ‘100%’ cloud.

The similar inferences in three otherwise unrelated conversations started me thinking more broadly about ‘100% adoption’. It IT, as in life, you never really need a Rolls Royce. You can aspire to the quality, appreciate its refinement, and in some cases you may be fortunate enough to actually enjoy it, but there is a point where it simply doesn’t make sense to pursue that level of luxury. Mostly you can get away with a Ford. Sometimes you can even make do with a second-hand Lada.

The same Pareto-like principle applies roughly throughout IT (much to the annoyance of just about every security pro I have ever met) – although the actual ratio may vary wildly, you can often get most of the benefit from less than a ‘100%’ implementation.

The phrase that sprang to mind for me was the same conclusion that I published elsewhere in the Responsible Cloud report, and the same notion that many IT pros live by, day in and day out:

It is important to look for opportunities, and do what makes sense

This should not just apply to cloud computing, but across all of IT.

Take, as another example, adherence to the IT Infrastructure Library (ITIL). Now, ITIL is a great framework, and an increasingly definitive reference for best practices in IT management. Data I have seen suggests as many as 60% of all IT organizations are committed to ITIL, and that implementation of ITIL (whatever that actually means) results in measurable and specific benefits in IT costs, staff and server efficiency, operational maturity, and more.

However, I also hear and read somewhat justified rants about how “ITIL just doesn’t work … ITIL is more 1960s than 2010 … it’s useless.” Yet the truth is, as so often, somewhere in the middle. In this too enterprises can definitely benefit from avoiding the dogmatic application of every single prescription. The same is true for other standards such as COBIT and ISO, or prescriptions from standards groups like the DMTF or NIST. All can deliver significant benefits with less than a 100% implementation.

It also applies in internal adoption of standard operating environment (SOE) components, like making singular (and often binding) choices between, for example:

• VMware vs. Hyper-V vs. Xen
• HP vs. Cisco vs. IBM
• HDS vs. NetApp vs. EMC
• Windows vs. Linux vs. UNIX
• iPhone vs. WinMo vs. Blackberry
• Solution suites vs. point products
• Mainframe vs. Commodity
• Physical vs. virtual vs. cloud

In all these cases and more, although standardization can have specific benefits, the greatest benefit to the enterprise does not always accrue from making an exclusionary choice; from committing to a 100% implementation. Most IT practitioners know that heterogeneity is the new standard – whether intuitively or grudgingly. They know that sometimes the best – or at least necessary – outcomes arise from providing multiple choices, fit to support multiple use cases.

Of course some areas are less flexible. You cannot, for example, pick and choose which parts of PCI, HIPAA, or Sarbanes-Oxley compliance would work best for you. Perhaps ‘close’ only matters in horseshoes and hand grenades, but for sure it doesn’t matter in legal compliance.

However, where possible, IT – practitioners, consultants, vendors, and analysts – need to stay away from dogma. We must avoid making any architecture, maturity model, or industry standard a religious ‘all or none’ battle. Important though they may be, these are not religious battles. These are IT decisions. Moreover, these are business decisions. So we need to keep the business goals in mind, and realize that sometimes a ‘100%’ implementation simply does not make sense.

As of Wednesday this week (2/24), I am now at one of the very best IT management software vendors, CA Inc., where I am leading product marketing for their — our — virtualization management solutions.

In many ways, this was an incredibly difficult decision. EMA is a truly excellent place to work, and the role of an industry analyst was fascinating and fulfilling. The people I worked with and for are some of the best minds in IT – always intellectually stimulating, and straight-out fun to be with. It was truly my privilege to get to know them all, and especially to help my clients and my team to be successful.

Yet this was also one of the easiest decisions I have made. I believe both virtualization and management deliver incredible IT and business benefits, and as virtualization becomes increasingly ubiquitous, management of virtual systems becomes increasingly critical. I have long considered CA a leader in physical and virtual systems management, and believe CA has a great opportunity to extend its leadership in virtualization management, by helping even more IT and business people to be even more successful. As a part of  CA now, I can not only be a part of that opportunity, but can be a significant author of that success.

Moreover, it allows me to indulge my passion for technology and my expertise in marketing in an in-depth, direct, and focused way, rather than the broad, ancillary, and essentially academic role of an industry analyst. I will be able to work directly with some the biggest and most successful companies and technologies, not just in the US, but around the globe. Plus, like EMA, CA also has some incredible minds who are some of the most fun people to hang out with too.

While some will see this a move (back) to ‘the dark side’, I have always considered analysts and vendors to be two sides of the same coin – helping IT to deliver business services in more effective and efficient ways. While some may say that I have ’sold out’ my integrity as an analyst, I have always considered my integrity to be a core and consistent value — and a non-negotiable one — regardless of my employer. While some may think I can no longer champion the best interests of enterprise IT like I did while I was an analyst, I believe the best software companies, and their best people, succeed and thrive specifically because they do exactly that.

As for this blog (and my Twitter feed), all my reasons for blogging and tweeting, and what I hope to achieve (both personally and professionally) with social media, are still the same as they were when I started. I therefore intend to continue writing and posting my personal opinions and insights about technology and other areas that interest me. After all, the areas I work with haven’t really changed, so I am still going to post about virtualization, systems management, data center operations, and cloud computing.

So although I cannot help but be informed by my current position and experience, my goal is to keep posting interesting and informed ideas, regardless of my employer. No doubt some people will stop reading — which is fine — but I still hope you will keep inspiring, contributing to, reading, commenting on, and arguing about these part-time musings of a full-time technologist.

The only significant and unique benefit of KVM for server virtualization (as noted by Sander van Vugt in our (virtual) debate on Xen vs.KVM Linux Virtualization Hypervisors) is that KVM is part of the Linux kernel. This ensures broad standardization, patch compatibility, simpler upgrades, and a low-impact on-ramp for existing Linux IT shops.

Yet this is a solution for a problem that does not really exist.

Large enterprises already run thousands of components, from services/daemons to drivers to applications, all as additions to various kernels. Maintaining one more (or even several more) non-kernel components like Hyper-V, XenServer, ESX, etc., is not a net negative. On the contrary, EMA data shows that virtualization actually improves the productivity of server administrators, and by an average of around 10% – up to 20% or more for best performers. For competent administrators with good lifecycle management tools, the time they spend to learn, test, and maintain hypervisors is a significant effort, but it is time paid back with interest.

On the other hand, many downsides to KVM are all too apparent.

It is easy to point to the lack of technology features and maturity in KVM – areas like live migration, paravirtualization, networking, isolation, performance, security, or a host of other  features which KVM (in some cases arguably) lacks. I have only some doubt that KVM will meet these low-level functional requirements eventually, but it will not be anytime soon. Yet they are essentially table stakes in server virtualization today.

The inherent dependency on Linux would also require a major shift in  platforms for the average datacenter (where Windows outnumbers Linux by  150:1), and a major investment in resourcing, training, and software. This is hardly an attractive proposition for a data center manager. Still, existing Linux staff will be able to pick it up, and could even have some success on their (relatively few) existing Linux platforms.

However, even if these weaknesses are overcome, KVM has a much more strategic problem – the gaping void in the KVM management ecosystem. There is almost no third-party support for KVM from management vendors. Even stated support from key partner vendors like IBM, HP, and of course Red Hat is basic at best. What’s more, EMA data suggests KVM will not foster a significant management ecosystem in the future, either.

EMA’s research on Virtual System Management showed convincingly how important management is to virtualization. Across 18 different management disciplines, almost all correlated with measurably better outcomes in metrics like MTTR, provisioning time, availability, VM density, migration speed, and more.

EMA’s new cloud research shows a similar importance. Applying mature automation and management disciplines to virtual systems is directly correlated with positive cloud outcomes like reduced CapEx, reduced OpEx, improved operational maturity and more.

Not surprising then, that over 80% of enterprises consider manageability an important or very important factor in their virtualization and cloud technology decisions.

Unfortunately, KVM ranks anywhere from 4^th to 10^th in enterprise preferences for virtualization and cloud technology providers. It comes behind first ESX, then Hyper-V or Xen (multiple implementations), often various UNIX hypervisors (PowerVM, Integrity VMs or vPars, Solaris Containers), and even z/VM. No enterprise demand means that management vendors have little incentive to support KVM.

In fact, in my conversations with management software vendors, most generally put KVM around 5th in line for support – which, realistically, means it is not even on the current roadmap. What’s more, for better or worse several of them have a vested interest in not supporting KVM (no points for guessing who).

This means KVM has little or no prospect of gaining third-party support for virtualization management tools like VM-aware backup and restore, VM provisioning, virtual resource management, VM configuration auditing, virtual performance monitoring, VM lab management, VM image control, storage management,network automation and more. The same holds true for integration with higher-level virtual systems management tools for virtual and physical data center automation and service management disciplines.

For any IT group, sophisticated management tools deliver many proven benefits. For larger enterprises especially, they are simply not optional.  Without even the prospect of a robust management ecosystem, KVM is simply a non-starter in most large-scale deployments. For my enterprise clients at least, it is certainly not a credible choice for x86 server virtualization.

Microsoft and Opalis

Today Microsoft Corporation (NASD:MSFT) announced a definitive agreement to acquire Opalis Inc., the leading independent vendor of IT Process Automation (ITPA) software.

IT Process Automation (ITPA) is a Data Center Automation (DCA) discipline that EMA defines as “the ability to automate and integrate the workflow of complex, multi-discipline IT management processes.” This automation can replace many manual, resource-intensive, and error-prone activities that typically cross multiple IT components, disciplines, and/or departments. ITPA delivers exceptional results including freeing up 77% more staff for strategic projects, providing more than 60 additional hours of system availability per year, and saving an average $500,000 more per year on staff costs than other Data Center Automation (DCA) disciplines.

This space has been gaining interest, both expanding and consolidating, for some time, as evidenced by significant development and acquisition activity from Novell (ZENworks, PlateSpin), HP (Opsware, iConclude), BMC Software (RealOps, Atrium), NetIQ (Aegis), Symantec (T-Logic, Altiris), and CA (Optinuity, Spectrum).

I think this is an excellent move by Microsoft. It will certainly make customers of both companies very happy. Microsoft and its customers gain an exceptional solution, in a discipline area that Microsoft was clearly lacking, and one which delivers many proven and exceptional benefits. For Opalis customers, it is probably a mixed bag. It will be a major change, but with Microsoft’s strength and stability, it is likely to be a positive outcome overall for Opalis customers.

This is, however, a huge blow for competitors, especially for the few large management vendors that have not yet acquired or built an ITPA solution or components, or whose own ITPA capabilities are less than stellar. For other large mgmt vendors with credible or better ITPA capabilities, this is both an opportunity and a threat. For mid-sized vendors that compete with Opalis or Microsoft Systems Center, and especially smaller vendors, this is a horrible result. Overall, most vendors will have to hustle to respond, although many will be unable to do so.

Meanwhile, Microsoft, Opalis, and their customers should be ecstatic with this deal. Few acquisitions are so clearly positive for the stakeholders as this.

You should be able to check out what the executives from both companies have to say in their blog posts:

• Blog post from Brad Anderson, Microsoft Corporate Vice President: http://blogs.technet.com/systemcenter/default.aspx
• Blog post from Todd DeLaughter, President & CEO of Opalis Software: http://www.opalis.com/blog.asp?id=1

Meanwhile, I will be expanding on the impact of this acquisition very soon with a full EMA Impact Brief. Keep your eyes out for that one – lots of significant implications for customer and competitors, without doubt!

Enter Intelligent Workload Management (IWM), which, according to the Novell press release is:

… Novell’s differentiated approach to Intelligent Workload Management [that] integrates identity and systems management capabilities into an application workload, thereby increasing the workload’s security and portability across physical, virtual and cloud environments

All I can say is … bravo Novell!

No, really. It is about time. Novell has exceptional capabilities in virtualization, automation, and service management; and it also adds critical capabilities for security management and compliance, especially around identity management.  These are all core values in what EMA calls ‘the responsible cloud’.

The EMA thesis, essentially, is that cloud computing has too many cowboys, and not enough sheriffs. Enter Novell, the “Doc” Holliday of the cloud landscape, with responsible capabilities for virtualization, automation, service management, and security and compliance.

IBM, Microsoft, Sun, and even Oracle might argue with Novell in some of its claims of uniqueness – after all, all of them have substantial capabilities in all these areas too.

However, regardless of some overreaching in their marketing, competitive threats, a nascent market, and gaps in actual product capability, Novell has an excellent opportunity to re-brand itself and deliver some exceptional capabilities to deliver on private cloud computing goals, and is as well positioned as any vendor to stake a claim to what they label ‘Intelligent Workload Management’.

Keep an eye out for EMA’s more detailed Impact Brief on this announcement. Very interesting stuff, without doubt.

Andi.

Good for them!

What I liked most was that they are trying to break down the barriers between systems and security management. Certainly this is something that I discuss regularly with enterprises – the need to stop focusing on silo-based management, and perhaps even more importantly, to stop pandering to silo-focused low-level managers. Almost all of the CIOs, VPs, and IT Directors who I talk with are critically aware of the problems these silos cause – including human errors, resource inefficiencies, security problems, and higher costs.

This is also a constant discussion I have within EMA, especially with the lead of our security practice, Scott Crawford – a brilliant mind on security (amongst many other subjects) who constantly thinks about security in ways I never could or would. We work and publish together on this topic frequently. Indeed, it has come up again in our latest research, which shows that security and risk management are a fundamental requirement for cloud computing – or what EMA calls the ‘Responsible Cloud’.

The upshot of all these conversations is simple – security management and systems management are not, cannot, and should not be completely separate. Not in human terms, not in processes, and not in technologies. Without doubt, anyone in a large enterprise who has ever tried to implement a patch, a configuration change, a firewall update, a software release, or a hundred other data center changes will attest to this in a heartbeat.

Of course (as Scott rightly pointed out when I last spoke with him about this), we will always need security experts, and systems experts – the two disciplines are not the same, and we will always need deep domain expertise in each. So I am not advocating complete convergence. But we need more software tools that provide integration and interoperability that allow these professional to work more effectively together.

While multi-function vendors like CA, Symantec, IBM, and others have the product portfolio to approach these cross-silo problems holistically, there are few ‘best of breed’ vendors thinking this way. Of course, Tripwire and the ever-inspiring Gene Kim (who I have sadly never met) spring to mind for me; so would Configuresoft (although now as part of EMC Ionix, hardly a niche vendor), and the indefatigable Dennis Moreau. Both inspire their teams, technologies, and customers by championing a fundamental understanding that systems and security cannot, at their heart, be completely separated.

(As an aside, these two seem like they would have been incredibly compelling arch-enemies in some ubergeek superhero genre – although I would never want to choose which should be the hero and which the villain!)

I must say that, so far at least, I don’t know the product design team from Reflex personally – guys like Hezi Moore, Aaron Bawcom, and Mike Wronski – as well as I do Dennis or Gene. However, I do know that they all have very credible security chops. Plus, one thing is clear to me.

They get it. They really get it.

And that in itself is a thing of rare beauty.

Deliberately designing functionality that addresses both security and systems management – like functional isolation, integrated access control, change segregation, granular audit trails, policy based management, and role-based access – into a systems management toolset is a rare feat, especially in startup and niche products. It is something I look for all the time, because my enterprise clients often demand it. Sadly, all too often I fail to find it – and I am not even a real security wonk! When I do, I am pleasantly surprised. When I see deep thought going into the security value of a systems management product, I am almost ecstatic.

Unfortunately, the challenge for vendors like Reflex and Tripwire (as it was for Configuresoft, and perhaps is still for EMC, Symantec, etc.) is to find customers that value this synergy. While most high-level IT execs understand this imperative, their holistic view frequently does not translate to many of their lower-level managers, or to many functional IT practitioners.

Of course, there are plenty of departmental ops managers and security managers who do get it. They strive to connect their teams with other groups, driving greater business efficiency and effectiveness as a result. However, unfortunately, many do not, instead focusing on protecting their small empires, walling themselves off from integrated management and cross-functional resourcing.

Similarly, many positive-minded individual technicians will actively seek out cross-skilling opportunities, recognizing that it makes them not just more useful but also more valuable, and more indispensable. However, many practitioners (both security and ops) can be just as bad as the most myopic managers (who they often work for), dogmatically eschewing integrated management tools and processes, seeing them as threats to their own personal domains of control.

Sad but true, best practices like breaking down IT management silos are not always adopted.

Fortunately, vendors like Reflex and Tripwire that have expertise and passion in both ops and security (and – shameless plug – trusted advisors like EMA, which is big enough to have experts in both disciplines, yet small enough that we still work together), are trying to break down these barriers.

And more power to them. They serve their clients much better by promoting the undeniable facts that security values are critical to systems management, and systems management is critical to security.

For example:

• Virtual appliances add an unknown operating system to the environment. It is typically a slimmed-down Linux distro, but you rarely know – it could be DR-DOS 6.2 or a pirate copy of Windows ME. This breaks any software SOE, ignoring top level decisions on OS stability, reliability, longevity, security, etc.
• Administrators have virtually no control over virtual appliance management. Management functions are required for any software, but virtual appliances rely entirely on a middle-man for proper OS, middleware, application, and database patches & upgrades, malware detection, performance monitoring, problem analysis, etc.
• Even when ad hoc management is possible, it is almost always manual. You can’t put agents on most virtual appliances, they don’t come with WMI, and most have only a GUI for management. So you cannot use standard tools or automation, which wastes admins’ time, risks audit non-compliance, and invites human error.
• Security is a particular concern. Timeliness of patches, effectiveness of hardening processes, zero-day threat response, malware protection, and so on are all at the whim of the vendor, and rarely disclosed to the customer.
• You pretty much have to pay maintenance. If you don’t, chances are you simply cannot keep a virtual appliance up-to-date yourself.

Of course, many of the same criticisms can be slated against physical appliances. I have even talked with one enterprise that will not deploy even physical management appliances because they would break the company’s hardware SOE (even though network devices, storage systems, and other ‘boxes’ are often just purpose-built appliances). However, with just an Ethernet cable connecting them to the enterprise, and a generally slimmer system profile, they seem to pose a lesser risk. They are also much simpler than virtual appliances, which add (in many cases unnecessarily) a layer of complexity and abstraction that physical appliances do not, by virtue of being encapsulated within a virtual machine. Moreover, the resources and effort to build a ‘real’ appliance is far greater than just slapping some software into a virtual machine, so physical appliance vendors seem somehow more committed, more reliable.

Is this distinction fair? Possibly not. But regardless of my own concerns, my research has shown that virtual appliances are the least-preferred of any form factor for management software, with physical appliances, niche software, and even software suites more preferred. Really, when the dreaded ‘framework’ is more popular than you, perhaps you really are an ugly duckling.

Which is not to say that virtual appliances are pointless. They are easy to implement, provide fast time-to-value, and are especially good for trials and POCs. They require little or no tuning, and the OS environment is often a bare bones install which is fast and efficient. Unlike physical appliances, they are easily scalable, and highly mobile. They can be deployed in seconds (maybe minutes) even to far-flung locations in regional offices with zero travel time and cost. And they allow even a sysop to deploy a new management server without getting the network, storage, security, or server teams involved. All of these are powerful factors in their favour.

I am also seeing, despite their potential issues, that several vendors are being very successful selling virtual appliances. KACE, for example, told me today that 26% of their total sales in Q3′09 have been of their virtual appliance, the V-KBOX; VKernel provide all their software in virtual appliance formats, and their Q3′09 sales were 205% up on Q3′08; Citrix is finding a remarkable early demand for their Netscaler VPX virtual appliance.  Meanwhile, IBM, Symantec, up.time, Reflex, SourceFire, and several others are agressively in or entering the market for management systems delivered as virtual appliances.

I also think that virtual appliances have a bright future – but in some ways I continue to see them as a beta version of what could (or should) come next.  By adding in capabilities for responsible and accountable management, they could form the basis of more fully-functional virtual service management containers. These in turn could form the basis of elastic, mobile, network-deployed, responsible cloud appliances that deliver complete end-to-end service management without regard to physical location or domain of control.

A couple of vendors are clearly headed this way, but even without this level of sophistication and maturity,  it certainly seems like vendors and buyers are increasingly embracing virtual appliances, despite their many potential flaws.

Perhaps I should too?

Andi.