Study: Buggy software costs users, vendors nearly $60B annually
Computerworld - WASHINGTON -- Software bugs are costing the U.S. economy an estimated $59.5 billion each year, with more than half of the cost borne by end users and the remainder by developers and vendors, according to a new federal study.
Improvements in testing could reduce this cost by about a third, or $22.5 billion, but it won't eliminate all software errors, the study said. Of the total $59.5 billion cost, users incurred 64% of the cost and developers 36%.
The findings were released yesterday in a 309-page study (download PDF) conducted by the National Institute of Standards and Technology (NIST), a federal agency that conducts extensive research on technology issues.
There are very few markets where "buyers are willing to accept products that they know are going to malfunction," said Gregory Tassey, the NIST senior economist who headed the study. "But software is at the extreme end in terms of errors or bugs that are in the typical product when it is sold."
A product of 18 months of research that included extensive feedback from end users, the study examined the impact of buggy software in several major industries -- automotive, aerospace and financial services -- and then extrapolated the results for the U.S. economy. The study was conducted for NIST by the nonprofit Research Triangle Institute in Research Triangle Park, N.C.
For instance, one case study in the automotive and aerospace industries involved interviews with 10 software developers and 179 users of computer-aided design, manufacturing and engineering systems and product data management software. That study found that users of these software design tools spent significant resources responding to software errors. Approximately 60% of those surveyed said they had experienced "significant software errors" in the previous year. The total cost in these sectors from inadequate software testing was estimated to be $1.8 billion.
Similar results were found in the financial services sector. In the case study of that sector, four software developers and 98 software users were interviewed. According to the study, developers agreed that an improved testing system was needed that could track a bug back to the point where it was introduced and show how it influenced the rest of the production process. The ideal testing infrastructure could remedy problems in real time rather than requiring developers to wait for the product to be fully assembled, the report said.
The total cost on financial services from inadequate software testing was estimated at $3.3 billion, the report said.
"No one thinks you can get all the errors out of software," said Tassey. Vendors are under pressure to getproducts to market quickly, and there are diminishing returns on testing: The more effort you put into it, the fewer the bugs that are found. "However, the general consensus seems to be [that] the current state of the art with respect to testing is poor and can be sufficiently improved," he said.
The study didn't propose specific actions for improving testing but called for the development of testing standards, noting that today's testing tools "are still fairly primitive."
The study also said that standardized testing tools, scripts, reference data and metrics, among other things, "that have undergone a rigorous certification process would have a large impact on the inadequacies" now found.
Read more about Management in Computerworld's Management Topic Center.
- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- Best Practices in BYOD: BlackBerry Enterprise Service 10 Manageability, security and support - these are just a few of the reasons BlackBerry® Enterprise Service 10 is such a powerful Enterprise Mobility...
- How to Migrate from BlackBerry Enterprise Server v4.0 or v5.0 to BlackBerry Enterprise Service 10 With BlackBerry® Enterprise Service 10, you can manage your entire mobile fleet through a single, unified interface. Find out how to make the...
Changing the Way Government Works: Four Technology Trends that Drive Down Costs and Increase Productivity
This paper discusses four technology-based approaches to improving processes and increasing
productivity while driving down department and agency costs.
- Shifting Gears: The Value of Customer-Driven Quality in Manufacturing In today's competitive manufacturing market, the customer must be the center of the quality universe. This paper details how manufacturers can improve customer...
- Live Webcast Best Practices for the Hyperconverged Enterprise Network To the Age of Constant Connectivity and Information overload
- Live Webcast On-demand webinar: "Mobility Mayhem: Balancing BYOD with Enterprise Security" Check out this on-demand webinar to hear Sophos senior security expert John Shier deep dive into how BYOD impacts your enterprise security strategy...
- Live Webcast Endpoint Backup & Restore: Protect Everyone, Everywhere Arek Sokol from the bleeding-edge IT team at Genentech/Roche explains how he leverages cross-platform enterprise endpoint backup in the public cloud as part...
- Testimonial: Cystic Fibrosis Trust Peter Hawkins, the Head of IT for Cystic Fibrosis Trust, discusses the role CommVault's Simpana software platform plays in improving the company's information...
- Increasing the Value of Your Reports and Dashboards Learn how incorporating other analytical capabilities such as predictive modeling and visualization can increase the value of your reports and dashboards by providing...