Considerations for Multi-Speed IT
Recently I posted about the "Big idea" I am focused on for 2015: Multi-Speed IT Capability. Let's drill down into some of the further considerations that come out of that.
Multi-Speed IT
To recap that introductory post, it's not enough to have a two-speed, black and white, DevOps-or-ITIL model. We need to have two approaches, Conservative and Nimble (my terms), as models that we use in our organisation. But I think any one project or development team can have some nuanced mix of the two depending on the business's needs. We may end up with three or more lifecycle structures in the organisation, as needed.
Different teams will have different needs; different system structures; different culture especially the level of acceptance of DevOps; different customers with different risk appetites. Any attempt to impose an A-or-B choice of lifecycle methodology on them is not much more helpful than trying to shoe-horn them all into one approach.
It is time we accepted flexibility in lifecycle design for IT changes. Instead of dictating how changes and releases must happen, we should set policy for the bounds in which they happen and the business rules that must be obeyed, and leave the stakeholders to it.
Those stakeholders include: the business owners of the service being created or changed; the builders; risk and compliance managers; change managers; and operations.
Multi-Speed Enterprise
I was asked if the enterprise also needs to be "bi-modal" for a bi-modal approach to work in IT. No.
Organisations are dealing with the same dilemma as IT, writ larger. On one hand they deal with the same competitive scramble, and on the other there is much talk of "mindfulness" and "slow business".
So they certainly could benefit from a bi-modal approach. But it is not necessary for Multi-Speed IT to work.
IT will always deal with conflicting needs of the business, whatever the philosophies they ascribe to. One part of the business will be demanding compliance, security, and integrity, whilst another urges rapid and radical change. That's life for IT everywhere.
Certainly as more enterprises embrace a multi-speed approach for the organisation, that does make it easier to align IT strategy, and to explain the considerations for a decision on a lifecycle design.
Governance and Risk
Any team adopting Nimble methods is asking for an elevated risk profile. This ought to be an informed business decision taken at the highest levels as a matter of policy, but often isn't.
Sometime I'll blog more broadly about the modern taste for increased risk in the name of agility or efficiency. I'd have thought recent economic events in the GFC put paid to that thinking that the rules of risk "gravity" can be suspended, but apparently not. You can get away with higher risk but only for so long. Repeated risk taking (say for example in high-frequency Agile releases) is only asking for probability to have its way. You can get away with it but not for ever. Everyone in IT should watch Aircrash Investigation as a job requirement. I had this conversation recently: I'm happy to visit Manila, but I'd be mad to live there for years on end.
So do what you can to escalate the issue of Nimble risk to the governance levels of the organisation, with as much accompanying advice as possible so that governors or senior management can make an informed decision. The decision is never on the CIO (or at least it should never be - it too often is). That is classic failure of Governance of IT. The decision is on the business, at whatever level of authority necessary to take the decision balancing risks and agility. The CIO should be trusted advisor, no more.
If the decision is to adopt the higher risk profile, then our job in IT is to manage that risk as best we can. Our job is to balance between Protect and Serve. We don't need to obstruct - we can collaborate within an organisational decision to go for it. That is the basis of the following tactics:
Trust and facilitation
Trust must grow in the organisation in order to empower selected teams to
- design their own lifecycle (with busienss owners, life cycle experts, and other stakeholders providing advice, assistance, and approval)
- build their own test and deployment infrastructure
- run their own lifecycle, especially to do their own risk assessment and production changes
If we don't trust a team to develop their own life cycle, then we should provide a standard one close enough to their needs. Likewise if a team don't want to develop their own life cycle, they should adopt a standard one.
Teams should own their lifecycle if it is not a standard one.
I chose the word "own" advisedly. Change Management should not own lifecycles (any more): they should facilitate, coordinate, monitor, and assure them. We need to get away from Change the Controller to Change the Empowerer.
Lifecycle design
There needs to be a CoE with expertise on lifecycle design that then takes the business directive and designs a lifecycle in conjunction with the team who will own it.
The design capability can sit anywhere. Personally I'd have it in the Architecture group but whatever: it could be Release Management, PMO, Development Management, whatever works.
Interfacing with core production systems
Ring-fence the particular system's production target where one team has adopted Agile, DevOps, or some other Nimble approach. If they adopt an elevated risk profile, it is Operations' job to maintain the risk profile of the other corporate data and applications unless instructed otherwise by the governors.
So try to limit the risk by separating out - ring-fencing - their production target.
Any data a Nimble method updates should preferably belong to their system, not be a shared resource of the organisation.
Any code they modify should belong to their system, not be a shared resource of the organisation.
Put them outside your security perimeter, or at least partitioned off by a firewall.
All interfaces to corporate data or systems should be via the same interfaces used by any external agency, usually APIs or Web Services, and treated the same way.
Give them their own little world to build and run in - and screw up - at their discretion.
And if you can't achieve that, then get a written waiver signed by the business owner of the group wanting Nimble, which agrees that you have explained the elevated risk to the corporate assets from rapid repeated change with reduced controls, and that business unit accepts all responsibility for loss or damage caused beyond their system.
Standard change
Give Nimble lifecycles as many standard change types as you can.
All organisations that have change control should have standard change (as defined by ITIL). If you don't have standard changes and you face complaints and resistance over change processes, it serves you right. I tell all my clients standard change is mandatory from day one in order to facilitate cultural acceptance of any new change management regime.
If you don't have standard change and a Nimble lifecycle is coming and you can't ring-fence it, you have a major problem. If you do have standard change, then you only have a serious problem. Work with the lifecycle implementers to find as many procedures as possible that can be categorised as standard change. This is of course an ongoing process: after they've pushed a few releases out there, you can start assessing the procedure for eligibility as a standard change. And then you can work with them to adapt their procedures to meet eligibility.
This isn't the place for a lesson in standard change, but in a nutshell a standard change is a type of change that is pre-approved. They still need to record the change but they don't need to seek permission every time they do it, just once in advance for all future occurrences of that type. The key elements (for me) are:
- Standard change types should be assessed on risk not scale of change. Big change can be standardised if it is low risk, and small changes aren't automatically standard ones until proven to be low risk
- Repeatability
- Fully documented and tested, especially including back-out/recovery procedure: this is the "standard" bit.
- Most of all: unambiguously defined. Before you open the standard change door, it has to be agreed and clear what constitutes that type of change... and what doesn't.
Automation
Automation of Agile change is a large chunk of what DevOps is about, at the technology level at least. Yes DevOps helps minimise the risk of Agile change, but it is not a silver bullet. It doesn't circumvent the need for the previous three tactics - it only reduces the risk somewhat. Three issues:
- Automation doesn't prevent errors, it just changes their type from many small ones to fewer more catastrophic ones.
- Automation is often inappropriate . Automation is expensive. We in IT always underestimate the TCO of automating our own processes.
- GIGO. Automation is just a tool. No tool fixes process, practices, roles or attitudes.
Outreach
Have an Operational Readiness programme
I've talked a lot about Dead Cat Syndrome, and the need for an Operational Readiness practice that bridges the gap between development and operations/production. A key principle of that practice is that the ultimate accountability is with the owners of production, and so it is Operations' job to seek out developers and talk to them not vice versa. Operations should have an outreach programme which offers advice, guidance and tools to help projects get systems operationally ready, baked in from day one of the project.
Project teams - including Nimble teams - should know what they are in for and what is required to get stuff live long before the day comes.
Operating Model
In this discussion I've focused on the Change Management aspects of a Multi-Speed IT Capability. But the different change lifecycles have implications for the whole operating model. If a team has a Nimble, DevOps-y lifecycle, they will need to take greater control of operations, support, and maintenance. At the front end, demand management, requirements definition, and service design will all be different too. So any change lifecycle variant implies an operating model variant.
But that's a discussion for another time. If not before, it will be the topic of discussion for the next Pink Think Tank, where Pink Elephant assemble a bunch of ITSM luminaries and thinkers (and me as producer) for a one-day workshop preceding their ITSM Conference in Las Vegas in February 2016. (Its their 20th conference so this should be a big one!)
Coming soon:
Cultural implications and culture change for Multi-Speed IT.
Made in New Zealand 
