Sizing Up Outsourcers

A new set of metrics helps IT managers measure providers' performance and reduce their own risk.

IT outsourcing contracts are expected to increase dramatically in number in coming years, forcing IT managers to face an array of puzzling choices in picking providers and monitoring contracts.

For example, if your company needs to outsource only one department or function, such as the help desk, would you know which outsourcers are best in that area? Could you compare the abilities of a number of providers with any objectivity beyond their bids? What criteria would you use to measure performance over the course of a contract?

Answers to those questions may soon be possible with a detailed outsourcing rating system being developed at Carnegie Mellon University in Pittsburgh with the help of several private companies representing both users and vendors. Called the eServices Capability Model (eSCM), the first version was announced in November. A second version was released this month for use by the public and beta testers.

The model ( uses 93 criteria, called "practices," to assess an outsourcer's capabilities in five areas: organizational management, people, business operations, technology and knowledge management. These practices are distributed across various phases of outsourcing. Independent evaluators trained in the model by Carnegie Mellon confer a rating from Level 1 through Level 5 following a comprehensive review.

Once audits are done in coming months, outsourcers are likely to publicize an eSCM seal of approval as a way to woo customers. The eSCM ratings are widely expected to be used the same way customers now use the Software Engineering Institute's Capability Maturity Model (CMM), also developed at Carnegie Mellon, to assess software development companies.

A version for use by customers to conduct their own assessments of outsourcers is being developed for release next year, according to officials at Carnegie Mellon. But even before then, an eSCM rating conferred by an independent auditor will be valuable for CIOs researching and gauging outsourcers, says George Brenckle, CIO for the University of Pennsylvania Health System, which has four hospitals in Philadelphia. "In order for the outsourcing arrangement to work, you have to have agreed-upon measures of performance and expectations," Brenckle says. "If you don't set those upfront, you get into a sort of push and pull, and somebody's left unhappy."

Much of Brenckle's 30-member staff is devoted to overseeing administration of a comprehensive IT outsourcing agreement with First Consulting Group Inc. (FCG) in Long Beach, Calif. Virtually all IT functions outside of telecommunications are part of the contract that started in January 2001 and could be extended beyond its five-year term. The deal saves the health system $19 million. Brenckle says he expects to use the CMM rating system to assess FCG's performance but hopes to substitute the new eSCM rating system.

"I am hoping the new system coming out will be valuable to me as an independent, unbiased metric on which to measure continuing performance of my outsourcer," Brenckle says.

Finding Room for Improvement

FCG underwent a pilot eSCM evaluation in July 2001, with evaluators from Carnegie Mellon scouring the company's records and interviewing its customers and employees. FCG has helped the school refine the rating process by attending strategy and feedback sessions and suggesting new wordings for practices, as well as practices to cut.

But the vendor also discovered areas where it could improve services, says Guy Scalzi, an FCG executive and a former health care CIO. "We're starting to beef up areas where we didn't do well," Scalzi says. One example is better planning for the first days of a contract with individual customers. However, "we got very good ratings on our people and experience," he says.

Jon Scarpelli, vice president at Ciber Inc., an IT services integrator in Greenwood Village, Calif., predicts that eSCM will be embraced by the industry. "Any tool a prospective customer can have to better understand and choose firms to help them be successful, that's a value," he says.

Jane Siegel, director of the Information Technology Standards Qualification Center at Carnegie Mellon, says she hopes the eSCM will help lower the risk of outsourcing. She points out that a study by Dataquest Inc. in San Jose shows that up to 25% of outsourcing relationships fail within a two-year period.

"With so much money going into outsourcing and such weak information and lack of standards, it's a little distressing," Siegel says. "A university can lead this in a neutral way," which can be especially helpful as corporate accounting scandals are making headlines, she adds. "Companies want to know if they have people doing things in an honest and reliable way."

Copyright © 2002 IDG Communications, Inc.

8 highly useful Slack bots for teams
Shop Tech Products at Amazon