ITIL’s dead elephant: CMDB can't be done
Note: this is the original "dead elephant" post from back in 2006. My thinking about CMDB has matured since then. Please see the articles (and book) in the sidebar of this article for a more complete picture of how I now see CMDB, and a much wider range of ideas why CMDB or CMS is - for most organisations - a bad idea.
This article has been podcast
CMDB can’t be done. Not as ITIL defines it. At least not with a justifiable return on the investment of doing it - it is such an enormous undertaking that any organisation attempting it is going to burn money on an irresponsible scale. The truth about CMDB is no secret. It is a “dead elephant”: a great putrescence in the corner of the room that everyone studiously ignores, stepping around it and ignoring the stench, because life will be so much simpler if they do not acknowledge the obvious.
Let us look at what is required of a CMDB:
“Configuration Management provides the foundation for successful IT Service Management and underpins every other process. The fundamental deliverable is the Configuration Management Database (CMDB), comprising one or more integrated databases detailing all of the organisation’s IT infrastructure components and other important associated assets. It is these assets that deliver IT services and they are known as Configuration Items (CIs). What set a CMDB apart from an ordinary asset register are the relationships, or links, that define how each CI is interconnected and interdependent with its neighbours.” [Ref 1]
“Detailing all of the organisation’s IT infrastructure components and other important associated assets”. I worked with a bank whose network and systems management environment managed 70,000 objects. That figure included no software (in-house or packages, applications or operating systems) and few logical objects like processes or services or owners. And they were not that big a bank. So I would say that a large organisation could well have a hundred thousand objects in a CMDB. Any organisation that doesn’t have thousands of objects to manage isn’t trying.
We hear that it is just a matter of getting the granularity right, and not including too much. On the other hand, the best working definition of what should go in is “whatever is managed by Change Management”. So ask yourself what isn’t. Should every PC be under Change Management control? Of course. What about the operating system on those PCs? Other software? If not, then you won’t be doing Release Management. What about the peripherals (keyboard, mouse etc)? No? You aren’t subject to occupational safety and health regulations? If someone suffers RSI and needs a trackball not a mouse, where will you track that if not in the CMDB?
So we will have between 1000 and 100,000 objects in our CMDB.
How will you populate the CMDB initially? The vendors’ silver bullet solution is auto-discovery. It can find out something about many things, but not everything about all things. It won’t help with disconnected devices, or financial information, or physical location. Ask how they go with finding UPS, PABX, factory machinery, or building security and cooling systems.
I worked on a project once that built a new retail system for an oil company in a moderate sized country. They had to populate 50,000 entities (tanks, trucks, warehouses, shops…). It took a team of – as I recall – three people for two years to capture and load the data. Another rule of thumb I use in Configuration Management is that any manually collected data is out of date before it is entered.
Maybe half the objects can be auto-discovered initially, but only half the data about them will be discovered. Warranty, contractual and other data still needs to be manually loaded. So expect between a few person-months and a few person-years to load the initial data.
How will you keep it current and accurate? By good tight Change Management which ensures the CMDB is updated whenever anything changes. How will you know if an error is made or someone subverts the process? The vendors’ silver bullet solution is … auto-discovery and comparison with the CMDB. See above for the limits of scope. Most tools don’t do this out-of-the-box: you will need to develop audit jobs to scan and report on discrepancies. And some manual report and review processes to pick up stuff the automated tools miss.
So expect a significant development effort to build quality control processes and tools.
Let us turn our attention now to “the relationships, or links, that define how each CI is interconnected and interdependent with its neighbours.” A single parent child relationship isn’t going to cut it here. We have relationships such as “is physically networked to”, “is responsible for”, “depends on”, “depends on but only at the end of the month”, “depends on to meet a gold SLA but can manage without it for silver”… Not to mention dealing with redundancy. Say we have seven web servers, equipped with load balancing and automatic failover. If one fails, what will be the impact on the SLAs? How many can we take out for maintenance without degrading performance?
What are the permutations between half a dozen relationships, with embedded business rules, and thousands – or hundreds of thousands - of objects? Capture those and keep them current! Maybe double your estimates.
But we are not done yet. Let us crank up the complexity by another order of magnitude: “comprising one or more integrated databases”. The probability of it being one integrated database is virtually nil. No vendor has technology that can manage the whole environment from .NET objects to telephones, so all CMDBs will be a federation of multiple vendor repositories.
The problem is that many organizations have multiple discovery tools to glean information from the same components. A federated CMDB must prevent data duplication, as well as be able to understand that different pieces of data gathered by different tools--an IP address, patch level, a host name--all belong to the same component. This process is called reconciliation, and experts say it's critical to a successful CMDB. They also say most reconciliation engines are proprietary. [Ref 2]
There is as yet no standard for their integration so all interfaces will be custom built. [For you geeks out there: ask how many integration interfaces support two-phase commit protocol to ensure transactional integrity when changes are made].
The CMDB is in its infancy. There are no standard definitions of what information goes into a CMDB, no schema for structuring that information, and no standards for integrating data from disparate vendors…. While the CMDB promises a host of benefits for the enterprise, it's also horrendously complex, lacks implementable standards, and is rife with proprietary exploitation. At present, there's no standard schema for the data that's supposed to reside in the CMDB… any CMDB implementation that aims to import and utilize data sources from disparate vendor tools will require manual integration. [Ref 2]
Vendors have rushed to fill that definition vacuum with their own implementations. For instance, Computer Associates created a data schema that's consistent across its own product line. This data schema populates the company's version of a CMDB. HP has been shipping a CMDB with its Service Desk that uses Web services to pull application information across HP's product portfolio. BMC Software's Atrium CMDB is shipped as a component of, or can be integrated with, eight BMC applications, including the IT Discovery Suite and the Remedy IT Service Management Suite.
However, all these CMDBs are essentially proprietary extensions to the vendors' own product suites. "There's no standard in the industry for how a vendor should build the data model inside the CMDB," says Colville. "There's a huge propensity to lock into one vendor for all of this." [Ref 4]
Some organisations will further multiply complexity by implementing a stand-alone “universal” CMDB which is synchronised with the other vendor repositories.
If you want something that meets the ideal DCMDB specification you are going to have to build it:
Every vendor claims to have a configuration management database (CMDB) or a CMDB strategy. Yet they are merely playing off Information Technology Infrastructure Library (ITIL) hype in this area and don't really have all the necessary functionality required for a true CMDB: reconciliation, federation, mapping and visualization, and synchronization. Rather, many are taking their domain-specific configuration repositories (such as for desktop or server configuration management, asset management, or help desk), adding one of these functions and calling it a CMDB. [Ref 3]
While the [sic] ITIL describes the processes associated with a CMDB, it says nothing about just how a CMDB gets built, how various tools are meant to feed data into the CMDB, how data should be structured inside the CMDB, and how various applications are meant to use that data. … "Customers are just trying to get a handle on what a CMDB can do," says BMC's Emerson … Enterprises that attempt to roll out a CMDB as a silver bullet for all their network management ills are likely to be disappointed. The difficulty of interoperability and the lack of standards mean a fully realized CMDB may be years away. [Ref 2]
My concern is that the convergence of architecture and process - and frankly often inflated vendor marketing - that now seems to be driving CMDB interest is complex and confusing. And CMDB technologies and standards are also very "early in the game." I am concerned that too many IT expectations are moving towards the notion of the CMDB as something that IT can simply buy to fix its woes. And this, of course, is both dangerous and false. While I am a big CMDB believer, I view it, like ITIL, as an enabler for more efficiency, improved compliance, better governance, etc. But it is not really a "thing." It is the beginning of a journey… [Ref 4]
So stack another development effort on to your estimates, to build the integration interfaces. Since these do not support two-phase commit, transactional integrity cannot be assured, so you will also need consistency reports, and repair policy and processes (mostly manual).
And now the sting: after all that I don’t believe CMDB is going to make that much difference to your ITIL processes. You sure won’t be able to automate any but the most basic impact analysis (the sort of thing vendors demo). The most sophisticated modelling tools on the market struggle to predict performance degradations, yet most SLAs put at least as much importance on performance as availability. They even struggle to predict availability whenever IP networks are involved, especially if the internet enters the equation (though complex intranets are challenging enough).
So after all that time and money a human is still going to have to look at a proposed change and make a judgement call; better informed than before, for sure, but still operating on imperfect information. Perhaps that money would have been better spent on a few nicer reports and exploratory tools, and another change management person, and a golf course for the staff.
In the past, companies wasted fortunes and diverted key resources for years trying to have one common relational database, and/or one common enterprise data model, and/or one repository of meta-data. They are doing it again trying to have one common repository of identity, or one repository of objects or Web Services. The sooner technologists and vendors stop peddling this kind of magic-fix crud the better off we will all be.
ITIL is about fixing the people and the processes, and only then implementing pragmatic tools to help them. How such an idealistic, bloated, infeasible, technology-will-solve-all-our-problems concept as CMDB got in there is beyond me.
CMDB is the only major example of ITIL describing what-should-be rather than what-is-and-how-to-manage-it, and it fails the test of common sense.
If you found this post useful, and you are a Facebook user, please Like the IT Skeptic on facebook
- 1 An Introductory Overview of ITIL, itSMF, 2004
- 2 Demystifying The CMDB, Andrew Conry-Murray, IT Architect, 8/01/2005
- 3 CMDB or Configuration Database: Know the Difference, Gartner RAS Core Research Note G00137125, Ronni J. Colville, 13 March 2006
- 4 The jury's in on the CMDB - or is it? CMDB adoption in the real world, Network/Systems Management Newsletter, Dennis Drogseth, Network World, 05/29/06