Get to the root of a performance problem with scorecards

Linking application info with financial or other business objectives lets you see how you're doing in key areas, and quickly.

At the 600-bed Maine Medical Center, information comes pouring in faster than ambulances rushing in with the wounded -- or at least it can seem that way. Hospital officials felt they needed a more efficient way to gauge their performance in areas including clinical outcome, patient satisfaction, doctor performance and safety, and then coordinate all of the data and make it available 24/7.

"We had PowerPoints, paper, Excel worksheets, and nothing was standardized," explains Peter Chingos, data analysis manager at the medical center, in Portland, Maine. Executives wanted to centralize that information and get data to senior-level administrators in a standardized way so it had the same look and feel, he says. The idea of creating balanced scorecards was tossed around, and, after observing an implementation at Boston's Brigham & Women's Hospital, Maine Medical Center also decided to deploy Strategic Performance Management software from SAS.

The hospital has created dozens of scorecards. Among the metrics: how often staffers wash their hands and whether a patient with both congestive heart failure and pneumonia is offered a flu vaccination. The scorecards allow hospital staffers to see how these changes -- compliance with best practices, process redesign and team building -- affect patient care and the hospital's finances. By checking progress on the intranet, staff members can see how their groups are doing on a monthly basis.

Today there are between 50 and 60 scorecards in use, each with some 25 metrics that give the ability to do subsequent drilldown to get charts, graphs and tables that provide more granular information, says Chingos. The hospital selects measures where improvement is needed, which makes the scorecards a tool for focusing employees on top priorities.

Maine Medical's leadership identifies these measures each year to reflect the hospital's quality- and safety-related strategic priorities. The current batch shows a focus on internal policies as well as regulatory issues, Chingos says.

It's an up-and-coming area. Business intelligence is scorecards' "parent on the software evolutionary tree," notes Ezra Gottheil, an analyst at Technology Business Research in Hampton, N.H. Performance management software is a refinement and a refocusing of business intelligence data so it is now matched up with goals and budgets, he adds.

Companies are using this approach to refine or outright change their current methods for measuring performance. Another way to use the technique is if the competition is gaining market share and they want to figure out what to do about it, Gottheil says.

Digging deep

Officials at Trican Well Service, an oil and gas well servicing company, found they were spending way too much time organizing and analyzing financial data and then getting the information into a forecasting model for each of the company's worldwide geographic regions. All told, some 80% of the time used for financial data was spent organizing the information, and 20% was spent on analysis.

"The immediate problem was replacing" the old budget-forecasting tool -- Pillar from Hyperion -- with something that would allow Trican to get information out quickly to the regions, says Randal Wichuk, director of finance corporate development at Trican, in Calgary, Alberta.

Executives wanted the different regions to take ownership of their financial performance so they could maximize profitability by looking at how to increase sales and decrease costs in each geographic location, Wichuk says.

After looking at performance management software from Cognos and SAS, Trican chose Hyperion's Performance Management Software and implemented it in September 2007. The software lets finance officials enter the data and run multiple scenarios to do very quick "what-if" analyses, Wichuk says.

He estimates that the tool has saved a minimum of six days each month in terms of loading data into the models and then doing the actual forecasting.

Keeping score

Now, Wichuk says, "what we're doing is measuring our key performance indicators." The software lets Trican analyze the data in multiple ways and drill down to the root cause of most issues.


One recent example: The ability to identify an area of the operation where sales were lower than expected. Once staffers drilled down further into the data, they discovered the company was losing market share in that region because salespeople weren't targeting the right customers, Wichuk says. Trican adjusted its prices for the region, "which helped us increase market share and revenues."

The real value of the software is its ability to see data in real time and conduct analyses, Wichuk says. "You're not gaining a value-add in terms of organizing the data; it's in terms of analyzing it to make quicker decisions" and react more quickly to the market."

Maine Medical's Chingos says the use of the balanced scorecards is voluntary, but in some areas, the metrics have been very high profile and have helped move the hospital in a more positive direction. For example, the hospital has a medication reconciliation metric that tells officials whether hospital staffers are comparing the medications a patient was on when he or she arrived to the medications that were prescribed during their stay.

"It's a step that would sometimes get done, but not always get documented well," Chingos says. Officials started measuring medication reconciliation about two years ago, and the results were "abysmal." It was in the 40% range, but using the scorecard to broadcast the issue has helped raise the number to the 90% range.

"The scorecard didn't do it per se; people did," notes Chingos -- but the scorecard helped staffers track that metric "every step of the way, and that motivated people." The biggest surprise for Chingos has been his end users' appetite for producing data in a more streamlined way, and there is a waiting list for other clinical areas such as the newborn nursery, the digestive disorder program, neurosciences and radiology that want to use scorecards. "It's driven demand beyond my expectations, and the demand for data in health care right now is huge. We're lucky and happy we have a tool that allows us to satisfy that demand."

How other IT metrics can fit in

In the purest sense, measuring Web site performance and availability may not relate directly to the notion of performance scorecards. Yet it does fall under the umbrella of trying to find the root cause of poor performance -- something that enterprises certainly measure and track. The New York Office of Temporary Disability Assistance (OTDA), for example, is in the midst of a year-long program to provide better services and assistance for families in need.

As such, the OTDA developed a Web site,, that allows clients to look for services related to nutrition, federal/state health insurance coverage and other forms of help. These constitute the services and money that people need when they're on disability, to pay for food, fuel and other essentials. "So there's a pretty high level of expectation around performance and availability because people's livelihood depends on the use of these applications," says CIO Daniel Chan.

The OTDA wanted to measure all of the activity coming in to its Web servers and then conduct different levels of analysis on how the public is using different applications, says Dan Donnelly, an OTDA consultant. The agency chose Transaction Performance Management from Precise, which enables IT officials to manage the availability and response time of some 27 internally developed applications.

"We'd like [users] to be able to complete a transaction in less than 15 minutes, and we're trying to understand how long it takes them," says Chan. The software also gives IT information about user behavior and demographics, so the Web site can be changed to allow users to find what they want more quickly. "Another component is making sure the Web site continues to be available and performing 24x7 and the response time has to be really in less than one second," Chan explains.

The Precise software lets officials proactively monitor the Web site throughout the day so they can detect problems before they occur. For example, last December the OTDA had a problem with one of its redundant servers crashing. That server could not recover cleanly, although it appeared to, which in turn put both Web servers in danger of failing. The Precise software discovered the situation and allowed IT staffers to do a controlled restart of the failed application server and both Web servers, completing everything without user outages. "As long as we could identify that the service was hung up within 24 hours, we could recover the system without having an outage," Donnelly says.

If an outage occurred, Chan explains, users would be forced to call the help desk at a cost of $25 per call. The myBenefits site receives an average of 25,000 hits a month, and Chan estimates that without the Precision tool, some 10% of users would call the help desk. Since much of the site's activity occurs on the weekends, if a server goes down on a Friday, it has the potential to be down all weekend, causing additional strain on the backup server, adds Donnelly.

"Performance management is about trying to find the root cause of poor performance, such as availability of a Web site or response time," observes Jean-Pierre Garbani, a vice president at Forrester Research. If IT develops an application that is supposed to have a response time of less than three seconds and an availability of 99.9%, and the level of performance is breached, performance management helps get to the root of the problem, says Garbani, in Naples, Fla.

The concept is broader than application performance management, however, which delves into code and bandwidth issues. It can also focus on capacity planning, to determine whether the available capacity of servers and storage is being exceeded. "The trend is to bring all of that together into a single dashboard," says Garbani.

Forrester projects that the application performance management software market will reach $2.7 billion a year by 2013, and Garbani calls it one of the fastest growing segments of IT management. While performance management issues are nothing new, they have become more complex as applications have grown exponentially in size, he says.

Adds Chan, "Until we had these tools it was difficult to have meaningful dialogue about issues because without data we had a tendency to do a lot of finger pointing about a problem in IT," since the data center is managed by another state agency. "Without tools we wouldn't be able to keep applications up."

Esther Shein is a freelance writer and editor. She can be reached at

Copyright © 2009 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon