I am hearing a lot about the rise of a concept called “devops” – a mashup of “development” and “operations”. I am not at all an expert in this area, but from what I can tell, devops is aimed at streamlining rapidly iterative application delivery to allow for greater development and business agility. Devops aims to achieve this by breaking down the barriers – human, process, and technology – between application development and system operations.
Interestingly, the concept is new enough that, as I write this, there is not even an entry for it in Wikipedia yet. I did find a blog by Damon Edwards (on Twitter – @damonedwards) very useful though, as he explains the age-old disconnects between application developers “throwing software over the wall”, and ops who are painfully resistant to change. James Urquhart (@jamesurquhart ) blogged very recently on the concept too , and again provided some very helpful content. Conversing online with them and others also helped me to formulate some more concrete ideas about devops – or at least some more concrete questions.
My interest was especially piqued when I understood how closely devops is connected to virtualization, cloud, and automation – my core interests:
- Cloud – Devops has antecedents in “rogue” developers (or developers from smaller shops) using cloud resources (IaaS, PaaS) for new projects, and will benefit greatly from cloud-based development and deployment, as cloud providers do not impose the restrictions of internal change-averse ops teams, and developers can essentially manage their own ops requirements instead.
- Virtualization – In-house devops (which needs more heavy lifting) is greatly assisted by virtualization, as virtual machines become the new base unit for application packaging, avoiding application rollout failures caused by incompatibility between the test and production environments (hardware, OS, middleware, etc.).
- Automation – In-house devops is also greatly facilitated by automation, which can use standard workflows to automatically provision and configure these complete application VMs, as well as backup and restore VMs, allowing complex composite application deployment and rollback at the click of a mouse.
Clearly devops has many very attractive outcomes – drive agile business, reduce delays, smooth application releases, deliver value faster. It is a very seductive idea. Who wouldn’t want it?
However, most of the writings I see about devops are really about dev, not ops. As a result, they don’t really capture the whole story of the application lifecycle. They justify devops as an antidote to the problems that ops are causing – slowing down release cycles, imposing arbitrary rules, screwing up deployments, killing developer productivity, hacking manual scripts and configs, stopping the business from being agile – but fail to recognize both the failings of developers that contribute to the problems, and the role of operations in delivering critical business outcomes during the application delivery lifecycle.
On the contrary, discussions mainly focus on how developers can sideline or change operations, positioning devops as the lone hero in the battle against inefficiency, as application developers fix all the problems (!) by controlling or automating key release management operations like provisioning, deployment, integration, patching, and software update. Meanwhile, ops are marginalised, along with their timesinks and roadblocks, satisfying the needs of an agile and rapidly changing business.
See – seductive, isn’t it?
Yet this seems to me (as a former op) fundamentally flawed, a development-centric neologism based on an incomplete understanding of the real purpose and role of IT operations, or of operations” history in the development of “agile” IT.
The way I see it, devops misses that target on how IT ops serve business needs too, and seems to gloss over “coal face” realities like:
- Who handles ongoing support, especially software update for the unrestrained sprawl of non-standard systems and components.
- Who ensures each new application doesn’t interfere with existing and especially legacy systems (and networks, storage, etc.)?
- Who handles integration with common production systems that cannot be encapsulated in a VM, like storage arrays (NAS, SAN), networking fabrics, facilities, etc.
- Who handles impact analysis, change control and rollback planning to ensure deployment risk is understood and mitigated?
- Who is responsible for cost containment and asset rationalization, when devops keeps rolling out new systems and applications?
- Who ensures reporting, compliance, data updates, log maintenance, Db administration, etc. are built into the applications, and integrated with standard management tools?
- Who will assure functional isolation, role-based access controls, change auditing, event management, and configuration control to secure applications, data, and compliance?
Because, if you have ever worked with both ops and apps, you know it is not going to be apps.
Now, in defence of devops, I am sure it is being implemented and conferring major benefits, especially in small organizations with little IT management discipline. I am sure the supporters of devops have some positive goals in mind too. What’s more, it is addressing a very real problem – ops really should spend more time on better processes and controls than in “daily deployment muck“.
However, devops should be a two-way street. As a former op, I know that the apps team have to pull their weight too, by addressing gaps like:
- Including ops during the design process, so applications are built to work with standard ops tools
- Taking ops input on deployment, so applications will go in cleanly without disrupting other users
- Working with ops on capacity and scalability requirements, so they can keep supporting it when it grows
- Implementing ops” critical needs for logging, isolation, identity management, configuration needs, and secure interfaces so the app can be secure and compliant
- Giving ops some advance insight into applications, especially during test and QA, so they can start to prepare for them before they come over the wall
- Allowing ops to contribute to better application design, deployment, and management; that ops can do more for the release cycle and ongoing management than just “manipulating APIs“
See, ops do enable business – and agile business at that – by ensuring that new applications coming into an existing complex environment are safe, secure, reliable, integrated, and responsive, regardless of how complex IT is, or how many moving parts there are. Devops seems to miss this important detail.
So I am sceptical of how devops will work in large, well-run IT environments with important and necessary operational controls, especially the > 60% of organizations that are committed to ITIL best practices (like formal and integrated management of change, configuration, release, assets, etc.).
After all, ‘agile’ does not magically obviate the need to identify and prevent bad changes, to reject apps that breach operational compliance, to ensure each new application adheres to standards, or to prevent uncontrolled sprawl of heterogeneous software.
I still have a lot to think about on this topic, and am trying to keep an open mind. But my best guess right now is that, for enterprises at least, devops either will not take hold or will not last. It seems most likely to be instead, at best, a transitory state on the path to a ‘new normal’. As with all “revolutions”, it has started outside IT ops, yet I expect will eventually co-opt and migrate wholly to operations in some form. Once the revolutionaries in development understand how many business needs besides agility actually require routine, process, management, and controls, they will back away from devops the same way they backed away from ownership in other IT revolutions – like the deployment of mini computers, desktops, and web applications.
If it does turn out this way – don’t worry. Operations will again dutifully take the reins, and clean up the mess that devops will leave behind. Because that is what ops do – they manage what they are given, and keep the business running, regardless of the mess that gets thrown over the wall at them.
In any case, whether devops takes root or not, hopefully we will all learn something about cooperation, automation, agility, and control. Because all stakeholders in the devops discussion – development, operations, and business owners – could benefit from that.