What You Measure Is What You Get

Computerworld - A well-written IT services contract defines many client-specific requirements. But all too often, the metrics necessary to measure and meet those requirements are glossed over during contract negotiations.
Worse yet, service-level agreements are rarely enforced as work progresses. Instead, buyers usually don't bother to look at the metrics they have negotiated until somebody complains. This is like balancing your checkbook only after a check bounces.
Metrics need to be specific, actionable and designed to ensure that the supplier meets the buyer's goals. Metrics that are highly specific but track the wrong things are useless. One example is a call center that measured the length of calls. Because the company was really interested in providing a very high level of service, a better measure would have been the percentage of problems resolved on the first call. Buyers often don't have sufficient experience defining appropriate metrics. Many contracts contain immeasurable and grandiose language such as "provide world-class support." Clearly defined metrics provide a baseline against which supplier performance can be fairly measured.
At contract time, many suppliers lack sufficient information about the customer's requirements to propose useful metrics. (In some cases, suppliers hope to get back price concessions by taking advantage of loosely worded metrics.) Challenge your supplier to share successful metrics from other customers.
The buyer is ultimately responsible for the success of any acquired product or service. Even when you outsource a function completely, it's still your responsibility to monitor the provider effectively. Research best practices. Interview peers and industry watchers to find out what metrics are commonly used and which vendors are meeting them.
To get the most out of your supplier measurement program, you should do the following:

• Prioritize your goals. Most buyers are interested in a combination of financial performance, service quality, operational excellence and the ability to meet future needs. Articulate and weight each goal before negotiating.


• Define metrics early. Metrics should be identified and negotiated as part of the initial contract, preferably by the same individuals who will be responsible for them after the contract is signed. But it's never too late -- even where contracts are already in place, negotiate metrics that will effectively measure success.


• Establish operational metrics that support your goals. Appropriate metrics will vary by company and by type of product or service being acquired. Select metrics that will accurately assess progress, and design metrics to influence the behavior you want. For example, monitoring "unit cost" metrics motivates your supplier to drive those costs down - especially if there is a bonus for doing so.


• Grade performance. Use a multilevel indicator of success, e.g., letter grades. If a supplier realizes halfway through the month that a target will be missed, it may quit trying if evaluated strictly "pass/fail." With a letter grade, the supplier can still get a B for the month.


• Use value-based metrics. Poorly chosen metrics allow suppliers to meet their contractual service levels and still disappoint the buyer. For example, some suppliers specify the number of people who will staff a function instead of specifying the expected output. Metrics such as response time per event and cost per service unit are inherently more useful.


• Use metrics that support your culture. Companies that normally operate based on data and numerical analysis can easily work with an extensive set of metrics. Companies that are based more on passion and energy may lack the focus for more than a small number of metrics at first. Stay away from complex measurement approaches until comfort increases.


• Define trade-offs. Prepare to pay more for higher levels of service. And be fair -- if you want to collect penalties for poor performance, also pay incentives for exceeding targets.


• Review performance regularly with the supplier. If the service you want is significantly better than the service you are getting, develop a service improvement plan with appropriate metrics and timetable. Service-level agreements form the basis for supplier governance and provide a solid foundation for clear communication. Metrics that are carefully specified and consistently reviewed can help manage expectations fairly and accurately. And they pay off -- what you measure is what you get.

Read more about in Computerworld's Topic Center.