SLAs: measuring an ITSM service as a black box is essential
Measuring a service as a black box is essential. Simply because it is impossible (in a practical sense) to discover and measure every link in a service chain, it is impossible to build a complete composite view of a service's performance bottom-up from the component CIs.
It is the boundary problem I have spoken of before. Something is always outside your range of view: telco links, proprietary hardware, "magic glue" [EDI, EAI, SOAP, UDDI, LDAP, CORBA ...], the internet, service providers, outsourcers. Web Services, SOA and grid computing are not making it any easier.
The only way to accurately measure performance and availability SLAs is to monitor the user experience, i.e. black box the service. All the discovery, analysis and component-monitoring tools are there to alert us and to sometimes tell us why a service is not meeting SLAs, but they never(?) give us the 100% picture by building a service view bottom-up.
(In the same way, the meta-view of the service can only be completely built top-down. The top layer links are all conceptual, perceived, existing only in wetware [people's heads]. Further down the stack there are some links that exist physically but lie outside the boundary. Both these types of links in the service-view have to be manaually built. This is why I say CMDB can not be fully automated.)
We are lucky that such black box service monitoring is not too hard. Measuring every single end user is impractical, but a good view can be achieved by selecting and measuring a representative subset of users by deploying an agent to their desktops. This gives us SLA reporting, and SLA monitoring and alerting.
It also overcomes the old problem of users complaining and IT saying "well, we can't see any problem" using their bottom-up composite view.
Finally, lots of performance issues are user perception. Users never remember how it performed six or twelve months ago. Their frame of reference is always the last week or month. If IT says it has improved over the year, they will insist that IT is not measurting something. End-point monitoring at the desktop gives objective stats of the measured experience to show the trend over time.
You see, there are some technologies the IT Skeptic actually likes :-D