Why Quality Pays

A software guru's message to executives: Your application development group can be a competitive asset - or a business disaster. By Watts S. Humphrey

While technology can change quickly, getting your people to change takes a great deal longer. That's why the people- intensive job of developing software has had essentially the same problems for over 40 years. It's also why, unless you do something, the situation won't improve by itself.

In fact, current trends suggest that your business will become even more dependent upon ever-more-complex software. This means that more of your people will work on software and that their work will be harder to track and more difficult to manage. Unless you make some changes in the way your software work is done, your current problems will likely get much worse.

The quality of software, its usability, and its timely development are critical to just about everything businesses do. The speed with which your people develop software can put you ahead of or behind your competitors. Software problems may have been frustrating in the past, but mismanaged software can now be fatal to a business. If your people don't produce quality software, testing times will be excessive, schedules will slip, customers will be unhappy, and revenue will drop.

Watts S. Humphrey, a fellow at the Software Engineering Institute at Carnegie Mellon University
Watts S. Humphrey, a fellow at the Software Engineering Institute at Carnegie Mellon University
To manage a software-intensive business, you must observe three management principles:

1. Recognize that you're in the software business. A senior vice president at Citibank once told me "we are a software business masquerading as a bank." It follows, then, that if you can't manage software, you'll have trouble managing your business. For example, delays in a software schedule can delay the delivery of new products and thus affect cost, revenue and profit.

2. Quality must be the top priority. In software work, quality problems overwhelm everything else. When quality isn't managed, entire software projects are unmanageable. There are ways to manage software quality, but they require proper training and disciplined software-engineering methods.

3. Quality software is developed by disciplined and motivated people. Software development is intellectual work, and undisciplined or unmotivated people can't do timely or predictable work.

In deciding how to manage software quality, consider these facts. First, while developing programs, even experienced software engineers inject one defect in about every 10 lines of code. The programmers aren't incompetent or lazy - they're just human. All humans make mistakes, but in software, these mistakes result in defects. This means that a modest-size program of 100,000 lines of code typically would start with about 10,000 defects.

Winning With Software, by Watts S. Humphrey
The second key fact about software quality is that the cost of finding and fixing defects increases at every step in the development process. For example, Xerox Corp. measured the time it takes to find and fix a defect at each stage:

• Code reviews, in which programmers analyze their own programs: three minutes.

• Code inspections, in which several programmers look for problems: 25 minutes.

• Module testing, in which programmers test small program modules: 32 minutes.

• System testing, in which the modules are combined so the whole system is tested: 1,400 minutes.

To appreciate the significance of these times, consider the time required to find and fix 10,000 defects. Using the Xerox data, the defect-removal time would be 500 hours for code reviews, 4,170 hours for inspections, 5,330 hours for module testing and 234,000 hours for system testing. Note that 234,000 hours is over 100 engineering-years of work. Clearly, selecting the proper defect-removal method makes a big difference in both software development time and costs.

The third key fact about software quality is that the person who developed the program is the one best able to find and fix its defects. This review takes about an hour for every 100 lines of code. Programmers who do careful reviews typically find and fix five to 10 defects per hour. Therefore, counting review time, the average defect-removal time in code reviews is six to 12 minutes per defect. With code reviews, the engineers can find and fix 10,000 defects in about 1,000 hours instead of about 4,000 hours with inspections, 5,000 hours with module testing or over 200,000 hours in system testing.

The reason testing takes so long is that a test reveals only the symptoms of a problem. Then the engineers must determine where the problem is in the system and its underlying cause. This is particularly difficult when there are multiple problems. Then the engineer must isolate the symptoms for each problem, identify the troublesome module, find the mistake and then fix it. After each defect is fixed, the fix must be installed in the system and the system retested. Because each of these steps can take an engineer an hour to a week or more, the time to find and fix a few hundred defects can be many months. This makes it hard to predict how long a project will take.

Not only does system testing take longer; it still produces software with lots of defects. The inspection strategy takes half the time and leaves less than half as many defects, while the review strategy takes a quarter of the time and leaves one-eighth as many defects.

Since code reviews by programmers are so effective, you might wonder why everyone doesn't use them. The reason is that good code reviews require disciplined methods and special training. The programmers must gather defect data and use the data to help them find and fix the defects. Because people tend to make the same mistakes repetitively, the data shows programmers what kinds of defects to look for. With few exceptions, programmers aren't trained in such quality methods and don't believe that reviews and defect data would help them.

Why not? It's because of the way they've been trained and managed. Starting with their first programming courses, they learn that the most admired programmers produce code at lightning speed and fix defects in testing. This fix-it-later attitude fosters poor practices. To get quality work, you must change this culture.

Fortunately, there are techniques for changing the culture and managing software projects. I developed two methodologies for the Software Engineering Institute:

• Personal Software Process, which replaces code-and-fix methods with a disciplined process that helps individual programmers learn about the quality of their work - without slowing them down.

• Team Software Process (TSP), which builds motivated teams of programmers who track and manage the quality of their work and who are rewarded for doing quality work.

v Companies that have adopted these methods dramatically reduce testing time and produce what is essentially defect-free software - on schedule or ahead of schedule - while reducing employee turnover. It takes about two years for the benefits to exceed the cost of implementation (mostly the cost of training).

When Boston-based Teradyne Inc. adopted the TSP, for example, its previous rate of 20 defects per thousand lines of code was reduced to one defect, and the company saved $5.3 million in the first two years. As a Teradyne manager once told me, "With the TSP, we're so far ahead of the competition that nobody will ever catch us."

Humphrey is a fellow at the Software Engineering Institute at Carnegie Mellon University, where he led the Capability Maturity Model effort and other work to improve software development. This article is adapted, with permission, from Winning With Software: An Executive Strategy (Addison-Wesley, 2002).

Special Report

New Tools, New Choices

Stories in this report:

Copyright © 2002 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon