Users Losing Billions Due to Bugs

Computerworld - WASHINGTON
IT managers have long known that software bugs cost money, and now, thanks to a landmark federal study, they know how much: $59.5 billion a year.
Nearly two-thirds of that cost, 64%, is borne directly by end users, with developers and vendors incurring the remainder, according to a 309-page report from the National Institute of Standards and Technology, a federal agency that conducts extensive research on technology issues.
There are very few markets where "buyers are willing to accept products that they know are going to malfunction," said Gregory Tassey, the NIST senior economist who headed the study. "But software is at the extreme end, in terms of errors or bugs that are in the typical product when it is sold."
The study, the result of 18 months of research including extensive feedback from users, examined the impact of buggy software in the automotive, aerospace and financial services industries and then extrapolated for all U.S. markets. The nonprofit Research Triangle Institute in Research Triangle Park, N.C., conducted the study for the NIST.
One case study in the automotive and aerospace industries involved interviews with 10 software developers and 179 users of computer-aided design, manufacturing and engineering systems and product data management software. Some 60% of those surveyed said they had experienced "significant software errors" in the previous year. The total cost in these sectors from inadequate software testing was estimated to be $1.8 billion.
Similar results were found in the financial services sector, where four software developers and 98 users were interviewed. According to the study, developers agreed that an improved testing system was needed that could track a bug to where it was introduced and show how it influenced the rest of the production process.
Not everyone agrees with the study. Cem Kaner, a computer science professor and attorney at the Florida Institute of Technology, said the report incorrectly blames poor testing for bugs. The problem is software design and development, he said. "Testing will expose a few of the weakness that are left," he said.
But Boris Beizer, an independent consultant in Huntingdon, Pa., said the NIST study is any that has ever been done in pinning down economic cost. Nonetheless, he said software is improving but also getting more complicated, "and that's driven by users."
Beizer said he doesn't believe the study points to a software quality crisis and said the market has forced many lousy vendors out of business. Users' best defense continues to be to "trash the companies that give them bad software," hesaid.
One vendor said the NIST study isn't taking into account productivity gains from software that's being asked to do a lot more. "They're not giving suitable credit for that," said Chuck Grindstaff, president of product life cycle management products at Electronic Data Systems Corp. in Plano, Texas.
The study calls current testing tools "primitive" and said standards, particularly those that lead to early detection of bugs, could reduce the economic impact by about a third, but it won't eliminate all errors.
"Patches are a hidden tax on users," said Andrew Jaquith, who studies the economic impact of security-related bugs at @Stake Inc. in Cambridge, Mass. The study "puts numbers around the age-old theory that it's better to catch things early than late," he said.

Read more about software in Computerworld's Software Knowledge Center.