The Pros & Cons of CMM

Computerworld - What's in a CMM rating?
Does hiring a CMM Level 5 service provider guarantee that an outsourced software project will come in on time and on budget?
Will a higher CMM rating automatically mean higher costs?
What impact does earning a rating have on software quality?
These are just a few of the questions confronting IT managers charged with contracting out an increasing volume of application development and maintenance work to lower-cost offshore outsourcers.
Developed by the Software Engineering Institute (SEI) at Carnegie Mellon University in Pittsburgh, CMM—short for Capability Maturity Model—is a set of rigorous standards for software development that's based on five levels. Of some 70 companies worldwide that have publicly acknowledged reaching the highest rating of Level 5, about 50 are in India, according to the SEI and Gartner Inc.
Not surprisingly, these Indian outsourcers aggressively tout their CMM rating, marketing themselves as top-notch developers with standardized, repeatable processes in place for delivering the highest quality software. Executing standardized processes also works to keep down costs, enabling Level 5 providers to pass on additional savings to customers, according to Sangita Singh, head of strategic marketing at Wipro Ltd., an Indian outsourcing company with U.S. headquarters in Santa Clara, Calif.
Research confirms that higher CMM levels correlate with fewer software defects (see chart below). But the highest CMM rating doesn't necessarily guarantee the greatest savings for customers. "The data on quality and maturity levels shows there is a definite improvement in costs and [on-time project completion] schedules," says Bill Peterson, program director for software engineering process management at the SEI. "But whether the supplier passes the savings on to the buyer, we don't know. That's more business than anything to do with the logic of costs.
"What we are saying is that as a Level 5, [suppliers] are better and they're able to charge more, not less," Peterson adds.
At the same time, a Level 5 CMM rating comes with no guarantees, and in some cases, it may even be overkill, experts say.
"CMM is a great discipline, and it is a great designation to have," says Bart Perkins, a Computerworld columnist and managing partner at Louisville, Ky.-based Leverage Partners Inc., which helps CIOs manage IT suppliers. "But the reality is that if an outsourcer is at Level 5 and the client is at Level 1 or 2, the client doesn't have the internal discipline to take advantage of the Level 5 provider's standardized routines."
Defining system or project requirements is a prime example. "With CMM, the entire requirements process is very rigidly defined. A Level 5 requirements document is very detailed and explicit and has metrics associated with it," Perkins explains. "But a company at a CMM Level 0 or 1 could have their requirements on the back of an envelope and no metrics. The Level 1 companies are lucky if they write out two pages."
The upshot, says Perkins, is that touting a CMM Level 5 rating to a Level 1 buyer "comes down to touting a feature that's of little value. It's like a car salesman in Alaska touting a car's great air conditioning. It may be great, but you can't take advantage of it."
Yet some companies, such as Farmers Insurance Group in Los Angeles, contract with Level 5 outsourcers exclusively, even though they may be unable to reap all of the benefits of doing so.
"The CIO dictated that we only do business with CMM Level 5 partners. It was a way of distinguishing the best companies from the rest of the pack," explains Alan Stanley, a program manager at Farmers.
"Beyond that, we don't take advantage of CMM. We tend to dictate how we want work done. We allocate work and processes based on what we do here, so I don't think we've really benefited from the CMM Level 5 side," he adds.
Helen Cousins, former CIO at Parsippany, N.J.-based Cendant Corp., says she believes that hiring a Level 5 outsourcer is a way to raise the bar for your own IT organization. "One of the things we gained out of necessity is the ability to more clearly define what we want," says Cousins, who is now CIO at Dex Media Inc. in Denver. "I've also noticed that when people working side by side are with people who are disciplined, it starts rubbing off."
But in a January 2003 report on the subject, Gartner analyst Partha Iyengar cautioned that users should also remember that CMM standards are descriptive rather than prescriptive, meaning that "they describe what must be done, rather than how it must be done." Consequently, a vendor can specify a certain way of executing a process that isn't the best possible implementation of that particular process.
In other words, Iyengar says, "CMM standards certification in no way guarantees that a vendor's internal implementation of these standards is best-in-class in any way."

CMM Checklist 1 2 3 Next » Recommend Digg Twitter ShareThis Email Print Send Feedback Reprints Jump to comments Project Management Additional Resources WHITE PAPER Do You Know the 7 Steps to Successful Data Migration? Approximately 60 percent of data migration projects overrun time or budget, while some fail completely. Download this white paper, "Enhancing Your Chance for Successful Data Migration," to learn the critical steps you need to take to execute a data migration project with minimum cost and risk to your business. WHITE PAPER Total Cost of Ownership of Server Computing Vs. PCs Read the Gartner research note to learn why the TCO of a server-based computing deployment used to deliver all applications to users is around 50% lower than that of an unmanaged desktop deployment. WHITE PAPER IDC White Paper: Linux in a Global Recession Economic downturns have a tendency to accelerate emerging technologies, boost the adoption of effective solutions, and punish solutions that are not cost competitive or that are out of synch with industry trends. This IDC White Paper presents the results of an IDC survey of 330 companies in Western Europe, Asia/Pacific and the Americas that measures the receptiveness to Linux and takes into consideration changing views driven by the disruptive economic environment that businesses face today. White Papers & Webcasts  Informatica 9 Launch: Transform your Business. Transform your world. LIVE Nov 10, 2009 12:00 PM ET    Architecting Business Intelligence Applications for Change: The Open Solution LIVE Nov 12, 2009 02:00 PM ET   Top 10 Habits of Highly Effective PMOs Download This Whitepaper Now!   Top to Bottom Performance Management Excellence at the City of Chicago View now. Your Solution for Delivering and Optimizing Applications Across the Enterprise Download this whitepaper. Brought to you by Citrix.   Effectively Implementing Datacenter Automation Effectively select and deploy the best datacenter automation solution today! Application Delivery Solutions Download this whitepaper. Provided to you by Citrix.   Aligning IT to Business: The Rising Importance of Application Delivery Networks Application Delivery Networking (ADN) will play a vital role in helping enterprises incorporate strategic technologies to achieve business initiatives. 8 Must-Have Features to Handle All of Your Network Demands Download this whitepaper. Provided by Citrix.   Strategies for Driving Down IT Support Costs with Lite ITIL Practices View this now! Forrester Consulting - Optimizing Users and Applications in a Mobile World Learn how to successfully deploy a WAN optimization solution that is specifically tuned for a mobile environment!   Mitigate Risk, Lower Costs and Improve Network Efficiency Create a stable IP network that not only meets today's challenges, but is flexible enough to also meet future demands. All White Papers | All Webcasts Computerworld Reports 8 Questions To Ask About Your Intrusion-Security Solution Computerworld Technology Briefings IT Service Management Computerworld Briefing - Analytics: The Art and Science of Better All Computerworld Reports Sign up to receive updates on the latest Management resources   Putting the Right Model in Place to Better Balance IT Supply and Business Demand IT leaders manage business demands and IT resources in many ways, but which methods are most effective? Tom Welsh, senior consultant of Cutter Consortium, compares two approaches and outlines their pros and cons in this Compuware-commissioned white paper. Get objective appraisals and real-world examples for both. Download this white paper  CIO best practices: Optimizing your Project Portfolio Management solution for greater business value A Project Portfolio Management discipline goes beyond ordinary project management to link IT with the business and help IT leaders better understand the benefits, cost, risks and value of all projects. Download this Compuware white paper and learn how to boost the business value of your IT organization. Download this white paper  Exclusive eBook: Six Project Metrics Every CIO Should Know for Application Delivery Success Most IT organizations spend their time measuring whether projects are on time or on budget. Read this IT executive guide to learn what key data tells you the degree of quality, client satisfaction and how well requirements were met. Make these metrics part of your overall application delivery management process - and your software can't fail. Download this eBook  IT Planning and Control: Unveiling the Interrelationships between ALM and PPM Listen in as Gartner Analyst Matt Light discusses two hot topics-- Project Portfolio Management and Application Life-cycle Management--and how you can bring these disciplines together to improve application quality and delivery. Get recommendations on merging the two methods and learn what type of results you can expect. Download this podcast  White Papers Top 10 Habits of Highly Effective PMOs Application Delivery Solutions Forrester Consulting - Optimizing Users and Applications in a Mobile World The State of PCI DSS Compliance at Organizations Today HP Technology Guide for Scalable Business Solutions Technology Brief: Technologies in HP ProLiant G6 c-Class server blades with Intel Xeon processors How to Secure and Accelerate Your Oracle Applications Enterprise Application Delivery: No User Left Behind Infonetics: WAN Optimization Appliance Market Highlights 1 Q09 WAN Application Delivery for Executives Your Solution for Delivering and Optimizing Applications Across the Enterprise 8 Must-Have Features to Handle All of Your Network Demands Faster, Cheaper and Easier to Maintain IDC Research Report: The Business Value of Consolidating on Energy-Efficient Servers Clipper Group Report: HP Provides Enhanced Options for Data Center Introducing the HP ProLiant G6 servers Optimize Performance of Datacenter to Datacenter Traffic Practical Strategies to Accelerate Business Applications Across the WAN Accelerate SSL Encrypted Applications Top 5 WAN Optimization Myths Sponsored Links Play NetApp's New Data Defense Game. How Do You Rank as an Innovation Leader? Download Recent Study. Play NetApp's New Data Defense Game. Extraordinary times demand Brocade Extraordinary Networks. No Payments for Up to 6 Months on Orders over $500. NetScaler VPX : Harness the Power of Virtualized Web App Delivery 64-page prescriptive guide to security, compliance, and IT operations. Streamline IT Costs. Boost Performance with WAN Optimization Enterprise Network & Server Monitoring Software. Cross over to Traverse... Keep your IT expertise up to date. Join the Intel Premier IT Professionals. Crystal Reports Server: More options. Not more money. Parallelism is not just for HPC. Create rich apps from desktop to device. Get the eval. Tolly Performance Report  "Citrix NetScaler with nCore Outperforms F5 BIG-IP" Save up to 61% plus Free Cheesecake Find IT jobs in the Healthcare industry on Dice.com With the NEW KODAK i700 Series Scanners get full speed at 300dpi! Not All QSAs Are Created Equal: What You Should Know Before You Buy The arrival of Serial Attached SCSI (SAS) marks a new era in storage scalability The AMD Virtual Experience Virtual Trade Show New DW Options and Price Points. View Teradata Platform Family Data Sheet. See how AT&T can help protect your network. 3 Innovations. 3 Free Downloads. Free trials of Windows 7, Windows Server 2008 R2 and Exchange Server 2010. Take the Netezza TwinFin TestDrive! Citrix NetScaler with nCore Outperforms F5 BIG-IP - Learn More New Analytical Platform Options. Download Data Sheet and More. Get more enterprise mobility value for up to 25% less. Unified Communications: Thoughts, Strategies and Predictions. Join the discussion. NYUs M.S. in Management and Systems. Offered Online and On-site. Increase UPS efficiency without sacrificing protection Crystal Reports Server: Single server access to reports & dashboards Create new opportunities. See business making progress now ESET NOD32 with ThreatSense(R) - Get your free trial now! Looking to add Unix/Linux functionality to Windows? Join the conversation about the hottest IT trends with Computerworld on Twitter. ITwhitepapers.com - Access thousands of white papers on 300+ technical topics. Leverage Your Cisco infrastructure for Superior Application Performance Learn about the AMD Virtual Experience Introducing: Project Icebreaker About Us Advertise Contacts Editorial Calendar Help Desk Jobs at IDG Privacy Policy Reprints Site Map CIO Computerworld CSO DEMO GamePro Games.net IDC IDG IDG Connect IDG Knowledge Hub IDG TechNetwork IDG Ventures IDG.net InfoWorld ITwhitepapers ITworld JavaWorld LinuxWorld Macworld Network World PC World The Industry Standard Copyright © 1994 - 2009 Computerworld Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of Computerworld Inc. is prohibited. Computerworld and Computerworld.com and the respective logos are trademarks of International Data Group Inc.