Measuring Up: Meaningful Metrics

Computerworld - Tim Stanley and the IT organization at Harrah's Entertainment Inc. in Las Vegas have won many awards for their world-class IT operation. Harrah's IT department has constantly monitored hundreds of details such as system uptime and project deliverables to track its performance over the past five years. It even links the success of its IT objectives to the company's earnings-per-share growth.

So you'd think that by now, Harrah's has pretty much nailed which IT metrics it needs to track, right? Think again. "I'm literally holding in my hand a revised list of metrics and scorecards for this year, and I continue to nosh through this with my team, year after year," says Stanley, senior vice president and CIO at Harrah's.

Tim Stanley, senior vice president and CIO at Harrah's Entertainment
Image Credit: Martin LePire/Klixpix

He isn't alone. IT leaders at companies such as DHL Express, Cendant Hotel Group and Kaiser Permanente say they continually wrestle with what they're measuring to gauge the effectiveness of their IT organizations. But by focusing on what business leaders are looking for, they're getting a lot more sophisticated at what to measure -- and what not to.

Beyond Rote

Best-in-class IT organizations have gone beyond rote metrics such as system uptime and help desk problem resolution. Some IT shops on Wall Street, for example, are studying things like how many stock trades or customers they're able to support for each IT dollar they spend, says Howard Rubin, an analyst at Gartner Inc.

But these kinds of metrics are much more difficult to use than the traditional IT time and transaction measures. One of the big challenges that many IT leaders face in developing effective metrics, says Rubin, is determining which activities are under the auspices of the IT department. For instance, some user departments at financial services companies have their own LAN administrators who are outside the purview of the IT organization, says Rubin.

"There's IT everywhere, not just in the IT department but in bank branches and on the plant floor," he says. The business activity supported by IT "drifts back and forth like the eye on the head of a flounder," he adds.

Metrics get even more complicated as IT organizations move to service-oriented architectures, explains Stanley. "In a classical IT world, you measure a particular application," he says. "But when these systems are all highly integrated, what do you actually track? The platform? The services?"