Where ROI Models Fail

Computerworld - Now that cost is king, more IT executives are relying on standard accounting methodologies such as net present value (NPV) and internal rate of return (IRR) to cost-justify IT investments to top brass. Sometimes they're using the models because the rest of the company is doing it.
IT leaders "should adopt what the business adopts, and that's fine if that's how the organization wants to view the numbers," says Audrey Apfel, an analyst at Gartner Inc. in Stamford, Conn.
Problem is, say Apfel and other pundits, return-on-investment models are usually incomplete and don't reveal as much as they should about intangibles such as an investment's impact on employee productivity, sales or customer satisfaction.
"ROI models fail because they become overly complex," adds Stephen Andriole, an MIS professor at Villanova University in Villanova, Pa., and a senior consultant for the Cutter Consortium in Arlington, Mass.

GM CTO Tony Scott

Andriole, who has held senior IT positions at Cigna Corp. and Safeguard Scientifics Inc., says that there are roughly 15 financial calculations that CIOs can use to tally ROI, "and that's part of the problem -- the more complicated the method, the more you have to feed the method rather than working the project."
That's why Andriole leans toward the use of more simplified ROI techniques, such as payback.
"At Cigna, we had all these complicated ways of collecting the data, with five people collecting data. And I said, 'Whoa, wouldn't it be more effective to have three of those people actually working on the project and just two people collecting the data?' "
Capturing Soft Gains
ROI models also aren't well suited to capturing soft benefits, such as customer satisfaction or employee productivity gains.
"If you reach a customer on the second ring instead of the third ring, how do you capture that? Are they more profitable? Are they happier? The human factor of computer-assisted work is neglected by the ROI model," asserts John Jordan, a principal at Cap Gemini Ernst & Young in Cambridge, Mass.
Further, ROI calculations typically fail to include the complexity costs associated with adding a new application -- such as additional hardware, storage or WAN/LAN equipment, and support personnel -- to an IT infrastructure, says Jordan. That's one reason he advocates asking another question: What's the cost of doing nothing vs. the cost of making an IT investment?
Jordan paints the following scenario: An IT manager is faced with buying five Unix servers in order to add new services to his company's Web site. The consideration isn't "just the hardware that you buy but the business capability you achieve in three weeks instead of three months," he explains. "That kind of thinking is hard to put a dollar value on."
The problem with relying solely upon financial techniques such as NPV or IRR "is that they don't necessarily capture all of the business benefits of an IT investment, nor do they help to evaluate all of the options that are open to you," says Chip Gliedman, an analyst at Giga Information Group Inc. Giga recommends that CIOs use options models, decision trees and other tools "to try to quantify and communicate the value of those options and the value of that flexibility."
Take a company that's about to install a new application that requires a two-processor server. You have two options, says Gliedman: "You can buy a two-processor server or a 16-processor cabinet and put two processors in it. The first option has higher immediate returns, but the second gives you the option to put additional processors in as they're needed."
A Computerworld.com survey indicates that it's common to try to calculate ROI for IT projects. Most of the 113 IT managers responding to last month's survey said their IT organizations do some sort of ROI calculation for major projects; only 6% said they do no ROI calculations at all.
But about half (48%) of the respondents agreed with the statement that "sometimes ROI calculations are a good idea, and sometimes they're not". Many IT organizations go to the trouble of doing the ROI math only for really expensive projects.
Sometimes, financial justifications have to be tossed aside when an IT investment simply makes good business sense, some IT executives say.
As General Motors Corp. Chief Technology Officer Tony Scott put it: "We have some areas where we use metrics to measure [business returns]. But in some instances, we have to go on faith."