The IT Skeptic - Comments for "Vendor marketeers stretch to reach out for ITIL"

Silo-ed monitoring as root-cause

John Worthington aka MySvcMon — Sat, 26 Jul 2008 13:56:30 +0000

Can't resist a few more comments here. I absolutely agree that [ "Many vendors hard sell monitoring tools which only address time to detect."];it can often take 8 hours to diagnose an Incident/Problem and 8 minutes to fix. So, your RCA/monitor tool should focus on diagnosis, not simply detection.

As far as multiple causation I have not seen tools that will do the kind of RCA that will tell you the root of your problem is lack of effective pre-production testing. Having a tool --- even a good one --- does not relieve you of your responsibility to dig deeper into the fundamental root causes; that's where lasting improvements will come from. So YES you need BOTH automation and good practice.

However, as the complexity of service infrastructures increases, the need for rapid isolation and diagnosis is becoming paramount; particularly in light of what is becoming an n-tier, virtualized mess.

For example, how long will it take you to diagnose that excessive disk reads by the media server are slowing down Oracle data base accesses? Or (adding VMs to the mix), that excessive disk reads by the user running MS Access is slowing down accesses for the user running Outlook & Word processing!

Service monitoring intelligence will not do your homework for you. You still need to apply good practices in the form of frameworks like ITIL, CobiT, et al.

But when you're up to your a__ in alligators, you'd better get some help. Silo based monitoring isn't working, and can actually be an inhibitor to the paradigm shift we seek.

So, why don't we dive deeper into multiple causation? Lack of pre-production testing may be significant; but if you can't quickly find out which layer of which component is the source of anomalies you never seem to get there.

Silo-ed monitoring is the root-cause. That should stir the pot a bit!

John M. Worthington
MyServiceMonitor, LLC

Multiple Causation

JamesFinister — Fri, 25 Jul 2008 08:33:09 +0000

The point about multiple causation is a good one. I've also always tried to push RCA back beyond the technical fault towards the kind of generic issues that impact a lot of apparantly unrelated incidents, for instance lack of effective pre-production testing.

The expanded incident lifecycle

Red Pineapple — Thu, 24 Jul 2008 16:51:12 +0000

All the usual suspects handle incidents by providing a time and then typing in some gobbledygook associated with the time. None enforces the expanded incident lifecycle. This lifecycle is the crucial link between incidents and problems as the trending of these times can highlight problems associated with insufficient workarounds plus time to detect, diagnose, repair, restore and recover. Many vendors hard sell monitoring tools which only address time to detect. In most case studies I have conducted that is hardly ever where the problem lies.
And while on the topic about the lack of RCA, additionally, most tools assume incorrectly that multiple causation is not required in managing problems.

The need for thought

JamesFinister — Thu, 24 Jul 2008 08:57:29 +0000

Handling major incidents as a a different process stream should be within the ability of most tools - and to some extent so should the extended lifecycle. In the early days of ITIL the classic question to a vendor was to ask "How does it handle problems?" and if the answer was "You change the I to a P" you would walk away.

Whilst the usual suspects can also manage the problem work flow (which of course is seperate again from the amjor incident workflow) I'm not aware of any that effectively actually support the analysis activity, that is the bit where you have to engage the crerative juices and actually DO the RCA.

Reminds me...

Red Pineapple — Thu, 24 Jul 2008 05:15:05 +0000

I once encountered a vendor who will remain nameless, who every time I asked for something that I had referenced or researched in the ITIL books, would answer with a "yes we can do that but it will cost you." I asked once, what would the cost be, and was rudely shocked at the price. My most efficient and effective tool remains a ball point pen and A4 counter book.

Getting to the root of the Problem....

John Worthington aka MySvcMon — Wed, 23 Jul 2008 22:38:44 +0000

Of course I immediately started to plug my favorite tool and had to revise my post....everyone has their personal favorites and I am no exception.

So I will simply say that if you can establish a cross-silo monitor that can look at every layer of every component --- across a distributed array of network, system and application components --- and automatically isolate which layer of which component is the source of an anomaly you'd be off to a good start.

As far as root-cause correlation, there are many approaches and I am not the geek here... I know of one that leverages a data flow and dependency based correlation which is embedded into the product architecture. :)

Of course, if the root-cause is a result of process, poor training, or otherwise unrelated to the service technology infrastructure you're on your own. So, what are you looking for?

You'll have to call or e-mail me if you want more info than that.

There's another problem though. I'm not sure that looking for tools that fit neatly into ITIL process boxes (Problem Mgt, Config Mgt, etc.) is such a great idea. In fact, ITIL's suggestion of obtaining an 'integrated ITSM suite' is kinda like saying all roads lead to nirvana....good luck. Don't get me wrong, I think it would be great but when at and what cost? Tomorrow's always a day away...

From what I've seen and heard from customers, the big gorillas product development is being driven by the marketing departments, who listen to the customer and in a panic-stricken fever rush to devour technology (and market share), which leads to an 'evolutionary' approach to product design and 'integration' that is more about account control and cost saving than solving problems.

You want ITIL? We do ITIL! You want root cause? We do root cause! Until you really try to do it, then it's

Burrrrp!

Look at what the tool does, and form your own opinion.

John M. Worthington
MyServiceMonitor, LLC

Thank you. However, now you

Michiel — Wed, 23 Jul 2008 22:04:32 +0000

Thank you.

However, now you ought to remove my post as well, or people might think that I refer to mr. Pineapples' post :->

Yes I removed it

skeptic — Wed, 23 Jul 2008 21:51:00 +0000

Yes I removed it. An interesting concept for a product but the post was a crude plug. See more here

Problem Management

Michiel — Wed, 23 Jul 2008 21:38:36 +0000

Hi Red,

I am not familiair with tooling that fills in your questions. IMO, this ought to be covered through an integral approach, touching tooling, people and process. I know of the existence of ATS (Analytical Trouble Shooting). Just google it.

Also, there has been a recent post where this subject was briefly discussed on the forum of itSMF international. Just look it up under Best Practices.

Regards,

Michiel

Survive-now.com

Michiel — Wed, 23 Jul 2008 21:34:00 +0000

Skep,

Why do you allow the above post [update: post removed] to remain on your blog? This is nothing more than rubish IMO. ITIL aligned? We can all name a few dozen of those. Extremely flexible? So is my uncles spine (and he is quite old already).

A few questions / challenges

Red Pineapple — Wed, 23 Jul 2008 18:48:35 +0000

Thanks for agreeing, skep!!! I want to touch on a single discipline in which I am interested in and that is problem management. I have a few questions and challengers:
1. Is there any IT tool that has problem management as a product attribute that does root cause analysis. I mean really management and facilitate root cause analysis and not just arbitarily populate fields in a form?
2. Is there a tool that actually uses and allows the expanded incident lifecycle to be populated and leveraged?
3. Is there a tool that has a different process stream for handling major incidents versus the rest?
The above is basic in problem management and if it is not there is the tool stretching the point about being able to assist in problem management?