EasyVista HIGH danger Crap Factoid alert
Chokey the Chimp issues a HIGH Crap Factoid danger alert for the EasyVista claim of "a stunning 271 percent ROI over three years with payback within 12 months ". Stunning indeed.
It is a white paper written by a familiar contributor to these Crap Factoid pages: Forrester. They seem to specialise in these factual-tap-dance case studies (though they don't have the market to themselves). It is not EasyVista's first Crap Factoid alert either.
The first thing for all readers of this claim to bear in mind is of course that another organisation's experience bears no relevance for your own unless they happen to be a very similar organisation trying to achieve something very similar with very similar levels of vendor attention and resources (let's bet on the likelihood of that last item). "Forrester makes no assumptions as to the potential ROI that other organisations will receive".
Which leads to the second point: anyone who pays attention to the results in a case study of one pet client of the vendor gets what they deserve. Unless of course they can also achieve pet-client status. To do this, you must agree to be a reference site (fodder for case studies such as this one), and you must be a site full of low-hanging fruit that even the vendor's product-jockeys can pick off. Alternatively you can just proceed with the vendor's tool and hope you turn out to be the one in ten where the tool delivers spectacularly, in which case the vendor will come knocking after the fact with offers of pet-clientdom: conferences, fact-finding trips, favourable write-ups (with a view to that next job)...
And third: the infuriating thing about these white papers is their pseudo-scientific aura. they present themselves as if they are research, as if the results have meaning. Which they don't. These vendor/analyst white papers are invariably scientifically meaningless. Like all the others, this white paper is nothing but unverified anecdote.
Putting aside those three obvious caveats that many readers will credulously overlook, let us examine the source of these claims, Total Economic Impact of EasyVista in more detail. (You have to register for it if you want to validate what I say here).
Getting a "stunning" result in this kind of study depends on stacking stuff into the benefits and hiding it from the costs. It is all sleight-of-hand. Here are the examples I found in a cursory examination of the white paper:
- "labor costs for ongoing operations are not included [in the costs] because the actual administration efforts required to maintain and run the new solution have decreased, according to the interviewed organization. This reduction in efforts is thus considered as cost savings and is taken into account in the Benefits section of this analysis". Slick! Hide it from the cost column and stack the hypothetical difference onto the benefits column. That'll really bring forward the payback period.
- "the organization now saves development time and costs as the service desk team can modify and change the workflows and other configurable items within the solution.". In other words the admin costs are now hidden in general overhead. Not to mention the fact that the system is now configured in an ad-hoc way by amateurs without - you can bet - any disciplines of requirements analysis, design, impact analysis, change control, training or documentation.
- "The organization paid a one-time fee of $158,600 for the perpetual EasyVista license... The organization pays an annual maintenance fee of $67,600". The smell of rat is heavy in the air. AFAIK, maintenance fees are 21%-22% of license fees. This organisation is paying 43%. Clearly they negotiated a 50% discount on the license fee, which has been recognised as a saving.
- "the cancelation of the maintenance contract saves the organization about $104,000 per year" The odds of negotiating their incumbent vendor down from $104k p.a. to $68k p.a. to match the offer on the table are pretty good, so this saving is probably illusory.
- At the same time "the organization avoided the upgrade of the legacy ITSM solution that would have been required. Due to the heavy customization of the legacy solution, it estimates that the upgrade alone would have cost $455,000" There is no detail at all on how this sum was derived, which is a shame as it accounts for a quarter of the benefits. As I've said in the past, I suspect they pulled it out of their own analyst. One clue we have is that it is referred to in Table 5 as "Avoided license upgrade and migration costs". it is a bit tough allocating migration costs to an upgrade of the incumbent tool but not mentioning them anywhere when costing the move to the new tool (see Table 3). Apparently the only deployment costs to move to a whole new tool were $39k on integration and training and $26k on internal labour costs. Oh really? "having integrated the remote teams into the incident management process and having introduced a self-service portal where both business users and IT staff can register incidents" with a total integration and training cost of $39k? Right.
- Fully a third of the projected benefits ($624k) come from three years of 'Development cost avoidance" at $208k p.a. This number comes from "The organization estimates that, on average, every process needs to be modified twice per year and that it saves about $13,000 in average for not having to call upon external developers.". Of course a more accurate estimate might have been to ask what they actually spent on modifying the product in the previous three years, but perhaps that number was uncomfortably close to reality.