Does ITIL V3 Intermediate certification need reform?

Memo

To: the ITIL qualifications/certifications/training industry

From: we the unrepresented punters paying for this circus, a.k.a. candidates

Subject: one more pissed off customer

The following post appeared on the Real ITSM group on LinkedIn

It has often been remarked that the point of the difficulty of ITIL exams is to test how well you know ITIL, rather than your ITSM knowledge, and I'm pragmatic enough to accept that.

However, the ITIL intermediate exams are the only exams I've ever sat where I have no idea whether I passed or failed. Does anyone happen to know what the failure rate is?

I think the structure of these exams is unconducive to a high pass rate. Only 8 questions leave a lot riding on so little. Plus means much of the body of knowledge will not be tested. For example, I just sat the CSI exam - not one question related to knowing the 7 steps of CSI!

Strangely there was also a question that I'm sure I will only achieve the whole 5 points because I happen to have read Kotter's "Leading Change".

Having sat a couple of the intermediate exams and done multiple mock exams, I've also noted that in half the cases the distractor answer isn't actually a distractor - it's just less correct than the other three according to ITIL.

Does anyone else think these exams need reform?

It was not an anonymous post of course, can't on LinkedIn.

I've copied the post over here with her permission because it will gain a much wider audience on this blog, and I'm interested in how representative it is of the views of candidates who have sat ITIL V3 Intermediate exams. Personally I haven't and currently don't intend to.

I know she is on the money with questions where the answers are of varying correctness and you are expected to make the same subjective evaluation as the examiners.

As for the issue of making eight random probes into your knowledge instead of a comprehensive survey, that has been the technique in other professions so it is not inherently wrong to do so. It does make passing a bit of a lottery: you either know everything or you got lucky enough to know the eight things they asked. And it is usually used in conjuction with some other more comprehensive general examination, not on its own (I'm thinking medical doctors' certification here: the verbal exam is such a random deep test, but it is not the only testing!).

Of course the idea of doing such deep testing with multi-choice is crap. I'd like to hear from an expert who says one can test Bloom's Level 4 or 5 (see the scheme, p 4)with multiple choice. I'm convinced this is without justification other than keeping down the cost of examination. (Note that CISA, CISSP and many other certs are also multiple choice - that doesn't make it right. ITIL started out with written exams and should have stayed there).

And I have heard say that the Intermediate questions are of an even lower quality than the Foundation ones.

Let's hear from readers who have sat (or taught) an Intermediate exam...

Comments

Another ITIL Intermediate skeptic

check out Aidan Lawes' latest broadside at ITIL intermediate certification:

have severe reservations about the style of testing at the higher levels, the suitability of the lifecycle and capability modules as defined and the quality of some of the learning programmes. There is also a big question mark about how the whole scheme is marketed and whether purchasers and consumers of training truly understand what they are buying.

And this really insightful remark

When one considers that accredited training organizations can enter their trainers directly into the exams without attending training courses and that huge swathes have done all the Intermediate exams in order to be able to teach them, one wonders whether the real number of [higher-than-Foundation] exam passes is distorted.

ATO sub-group in ITIL Qualification board

The good news is that there now exists an ATO subgroup who advises directly to the ITIL Qualification board, still the real clients are not yet heard, but I hope the ATO members can have some influence......

sheep guarding

"still the real clients are not yet heard" indeed. Allow me to paraphrase: there now exists a sub-group of wolves advising the alpha pack on sheep guarding

The world of ITIL training is changing

Get out and look around. ITSM and ITIL are entering the tertiary education mainstream. The days of the existing amateur-teachers IT-centric reinvented-consultants ATOs are numbered. Charles Sturt University is one exemplar. Another is the comment at the bottom of Shirley Lacy's latest blog.

Practical scenario

When I heard of the eight questions for the intermediate exams of the capability stream I was not sure if it is a good idea and how it could work.

But after reading the scenarios and questions, i have to say that I'm surprised how practical they are. It's not just about knowing a couple definitions as in the foundation exam, but about real world scenarios. so far, it's not a bad thing.

Good exams, except when they're not

i agree, in principle the 8 question intermediate exams can and should cover the material at a much higher proficiency level than the Foundation exam. EXCEPT when the question writers and reviewers get lazy, then you get lousy questions that include pure memorization based on the book, not based on evaluation or analysis. Had 2 of those in a SD test. Also had 1 other that had ZERO relationship to the scenario (about service catalogs, and SLM including SLAs and OLAs) and the question was about something else altogether (tool selection, supplier management, and SD activities). In other words, 3 out of 8 questions were horrible! Still passed, but only because I REALLY know the material and happened to read the SD volume to prepare and had reviewed section 3 (Activities) and 4 (Processes) the day before the test.

Further, there is no recourse, no review, no way to ask what might have been missed. No breakdown of areas covered versus scores, etc.

In other words, it's the creation and followup procedure(s) that needs reform, not the format or conduct of the exams.

David

Did 'm all

Well, I recently did the nine intermediates in 3 days, failed one an then did that one and Managing Through The Life Cycle a week later and succeded both. As a trainer I did not have to take the training myself and frankly I think that having followed a training would not have helped me much. I've tried to read the books, but after falling asleep every two pages (the best anti-insomnia being Service Strategy) I moved to another way of studying. I went through each syllabus, read for every chapter introduction, purpose/goals, benefits for the business and took a sneak peak at the tools paragraph. After the first three exams I even did not look anlymore to the mocjh exam.

A few observations ( and tips for thos who have to do them!) from doing the exams:
- I did not perceive many differences between the LifeCycle and the Capability exams, every time nearly the same approach
- the english in the questions is quit complex for a non native speaker
- there were some errors in some questions as of words (or part of the sentence had fallen away). I did signal the errors to EXIN (where I did the exams online)
- when I re-took RCV it was exactly the same exam as a week before so I doubt wether there are meny versions) I passed with 100%
- the scenario's are alle very much alike (you are the valued expert who is to choose between 4 possible options)
- I answered nearly all question by comparing the answer possibilities giving them + +/- or - (if there was clearly non-sense in the options). Most aswers are composed
of 5 to 6 statements which are often repeated between the different answer possibilities. It's just a matter of seeking the different statements within two of the
answer options and deciding which is the differenciator. I think most of the times I found the 3 and the 5 point answer ruling out the others.
- I prefer the ones where the scenario is given on the screen and the answer options on paper (easier to rule out options or to detect similair options)
- The MALC exam had three questions who had absolutely nothing to do with ITIL. Anyone with a good knowledge of general management theroies could have
answered these.
- I took most exams in less than one hour, sometimes being even surprised I passed!

And now: would I prefer to hire a V2 service manager or someone who got his ITIL Expert via V3? No doubt! I'd hire the first!!

I do not now who, how and why decided to create this current certification scheme with the current exams but I think the work should be done again and this time better!
In my opinion the currect set of certifications is a devaluation of ITIL certification. The books contain a lot of good things which merit to be explored and discussed during a training.

I would plea for a consultant stream and a practitioners stream with mandatory at least 50% of excercise time in an instructor led class with an assessment at the end (parlty MPC, partly written and partly assessemnt by the instructor and an independent person). If we want to take ITIL seriuously exams with only MPC questions should be restricted to foundation level.

The current level of intermediates can be done with online training, the Intermediates I'm proposing cannot!

Intermediate exams

We teach students two things in our classroom and online ntermediate training programs

1. The information they need to prepare for the examination
2. How to approach a complex multiple choice examination so they will select the best answer for the question being presented.

The combination of these two approaches has enabled our student population to score very well on the exams.

Intermediate exams

The above comment is spot on. I can teach students to pass the exam without them needing in-depth knowledge and undertanding of the content. Well that is sort of OK...but it is the verification of attendance that concerns me. A simple certificate to say you have attended the class. And the same for on-line. Bring back the in course assesments please.....It was the activities against a case study that really put meat on the bones of the Practicioner courses (yes, sometimes low grade meat!). The assumption from employers is that if they send someone to one of these courses that can now implement change based on their newly aquired knowledge! The courses should give them some tools to do this......

Intermediate Exams

Andrew,

My comments were directed at both our classroom and online programs. We have had several students sit our self-study online classes and have great success on the exams. I just got the numbers on how many exams have been administered to date across all EI's and our online students make up a good portion of them.

The key to our success for the online self-study programs is that we have recreated the value of the classroom (lectures, courseware, mentoring, study guides) online with the expense of the classroom venue.

EXIN publishes pass rates.

ITIL®V3 Intermediate
94 ITIL® V3 Intermediate: Continual Service Improvement - English
74 ITIL® V3 Intermediate: Managing Accross the Lifecycle - English
97 ITIL® V3 Intermediate: Operational Support & Analysis - English
100 ITIL® V3 Intermediate: Planning, Protecting & Optimizing - English
68 ITIL® V3 Intermediate: Release, Control & Validation - English
87 ITIL® V3 Intermediate: Service Design - English
96 ITIL® V3 Intermediate: Service Operation - English
91 ITIL® V3 Intermediate: Service Offerings & Agreements - English
63 ITIL® V3 Intermediate: Service Strategy - English
79 ITIL® V3 Intermediate: Service Transition - English

Service Strategy.....

I recently passed it with a score of 40/40 - It is my 5th intermediate exam, I'm thinking they're a bit like the Times crossword puzzle - once you know the style of the questions / answers they become easier!!

Looking forward to the Managing across the lifecycle course and exam....

Received 2 certificates in the post from the ISEB by mistake!!

Recently sent off to the ISEB for my ITIL pins for V3 - received a shiny new V2 managers red badge by mistake too!!

Also on a recent course the trainer remarked that the ISEB had given him a mark of 37 /40 for an exam which is impossible!!

No faith in the ISEB....

T

Pass rates

There's one problem with these EXIN pass rates, they're for 2008. Some of those exams were only available to trainers at this stage and all of them will be skewed by the number of trainers taking the exams (not that we all passed them).

Intermediate exams

We have been running several Intermediate classes (classroom, virtual classroom and self-study online) lately with a very high pass rate...we actually has one student take the OSA exam and get a perfect score.

probablility is greatly increased

Congratulations. Sorry to be a skeptic but with only eight questions the probablility is greatly increased of FLUKING a perfect score.

passrates

Generally have seen the passrates sit between 60-90% depending on the size and program.

More of a concern is the transparency of the exam reviews. We have heard of entire groups failing, ask for a formal review, then hear back that as a result everyone passed. I passed everything but RCV the first time around, scoring around 23 I think. I took the exam again a 2 months later without any extra preparation and scored 40.

I call BS and propose a vote of no confidence in Chancellor Valorum...

Bloom Level 4 or 5 and Test coverage

I don't get upset with coverage for any of the exams (and I've taken a number on both capabilities and lifecycle). There's a pool of questions for each scenario and a pool of scenarios. You get what you get. :-) I suspect there's an automated selection process that guarantees so degree of coverage of the subject matter (e.g., I only had 1 question with a focus on Change Management though others touched on it).

re: reform, yes, but for different reasons than the post you quoted.

I've had questions on the exams that were more about memory than analysis and evaluating (Bloom level 4 & 5). What is the right sequence (A) 8-1-3-4-5-6-7-2, (B) 1-8-5-4-3-6-7-2, (C) 1-8-4-5-3-6-2-7 (D) 8-3-1-4-5-7-6-2

The question didn't need the scenario and doesn't follow the Bloom taxonomy. IOW the reason for reform isn't coverage, that's luck of the draw, it's horrible questions.

David

Syndicate content