Software quality is still a work in progress, offshore and in the U.S.

By Mark Willoughby

Computerworld |

The link between software quality and security could boost businesses' use of software developed offshore as they battle against the worms and viruses that exploit software defects and cause billions of dollars in damage.

"There is no such thing as defective and secure software," said Watts Humphrey, a fellow at Carnegie Mellon University's Software Engineering Institute (SEI) and a former IBM software engineering executive. The SEI was formed in partnership with the U.S. Department of Defense in 1984 to improve the quality and productivity of software engineering.

Software quality efforts, such as the SEI's Capability Maturity Model (CMM), can reduce the number of defects and thereby improve security, Humphrey said, and offshore software developers, particularly those in India, were early adopters of CMM software quality methods. Software quality processes are beginning to offer tools to discover hidden malware, one of the major threats to software integrity.

"Half the Level 5 CMM-rated organizations in the world are in India," Humphrey said, adding that the other half of the top-rated software quality operations are mostly dedicated to U.S. government programs. Software companies in the U.S. have largely been slow to seek the lengthy and costly CMM certifications. The SEI's 2002 annual report states that "commercial software products today are riddled with defects ... that render them vulnerable to cyberattacks."

"Too often, management sacrificed software quality for adding more functionality and getting the product released early," Humphrey said.


	Mark Willoughby, CISSP, is a 20-year IT industry veteran and journalist with degrees in computer science and journalism. For the past seven years, he has tracked security and risk management start-ups and is a managing consultant at MessagingGroup, a Denver-based content development specialist.

Value seen in software quality testing

Some of the major Indian software development companies, including Wipro Ltd., Infosys Technologies Ltd. and Tata Consultancy Services, recognized early the value of software quality in differentiating their services to help overcome barriers to entering the U.S. market. They began the certification process more than 10 years ago. It was a strategy borrowed from Japanese auto manufacturers, which used superior quality to enter the U.S. automobile market. The SEI has a policy to not release any information about software certification audits, leaving it to the individual companies to publicize their CMM efforts.

"Security is primarily a design problem, not a coding problem," Humphrey said. "To do good security, engineers need to be using good design methods, and engineers historically have not done a good job of design. It's almost impossible to evaluate software code for security defects. You've got to look at the design.

"CMM guides you in setting up a good configuration management system," with the entire software development system and source code being documented, which is useful in finding many security flaws, Humphrey noted. However, CMM 5, the highest certification, doesn't "extend traceability to individual programmers" so that a specific developer can be held accountable for defects in his code, he added.

Making developers accountable

The SEI's next big effort to improve software quality and security is Team Software Process (TSP), which brings traceability of specific code modules to individual programmers, Humphrey said. Indian software companies and a few U.S. developers, including Microsoft Corp., are aggressively implementing TSP.

"The team knows who owns what code. It's effective for identifying and fixing defects as well as for finding security vulnerabilities. Good teams can find more than 99% of their defects," Humphrey said. "With TSP, we train engineers to follow high-quality [development] practices. They focus on design, cleaning up requirements, thorough analysis and studies of their design and code before going into test."

According to Humphrey, the results are dramatic. Documented code defects in a CMM Level 1-rated organization are 7.5 defects per 1,000 lines of code, which drops to one defect per 1,000 lines of code in a CMM 5 organization. Introducing TSP lowers that to 0.06 defects per 1,000 lines of code.

Microsoft is an early adopter of TSP, and the SEI conducted a special TSP security workshop in Redmond, Wash., last year. "I am very impressed with the Microsoft people and what they are doing," Humphrey said. "Once the engineers get it, they love it."

Making Microsoft developers responsible for the quality of their code is the main focus of the company's Trustworthy Computing Initiative, according to Steve Lipner, Microsoft's director of security engineering strategy.

Microsoft focuses on the parts of the software assurance process that have the potential to increase the number of code vulnerabilities. "When a vulnerability is discovered, we do after-fact postmortems to determine how that vulnerability got introduced. We look for similar vulnerabilities elsewhere in the code to see if other problems will be encountered," Lipner said.

"Security has been discussed primarily from a perspective of improving the security of the code from attack. We know we've gotten better, and we know that perfection is not there yet," he said. "Good security requires constant vigilance, training, research and updating of tools."

The nagging buffer-overrun issue

A particularly thorny security problem is buffer overruns, Lipner said. Buffer overruns have been the path of least resistance for most exploits targeting Microsoft software.

"The fact of the matter is a buffer overrun is not a single thing. Over time, we build automated tools to either detect or block the exploitation of buffer overruns, but both our people and outside security researchers are always finding new types of buffer overruns that haven't been exploited," said Lipner.

Microsoft also is putting the Windows operating system through the Common Criteria security profile evaluation. A battery of rigorous tests provides a baseline of assurance through a security profile. However, it doesn't specify the actual security of the software being evaluated, nor does it identify the software's country of origin. The International Standards Organization adopted the Common Criteria as ISO 15408 in 1999 as the international information assurance standard.

Windows 2000 has a Common Criteria rating of Evaluation Level 4, considered the highest rating for a commercial off-the-shelf product, since it doesn't require a source-code evaluation. Windows XP and 2003 are undergoing Evaluation Level 4 testing now.

Mary Ann Davidson, chief security officer at Oracle Corp., said the globalization of software development dictates worldwide development processes. "No matter where you build the software, you must have a culture of security with good internal processes," she said.

"At Oracle, we've been meeting the most stringent [security and development] requirements for years. We continually look at ways to make our code better for economic reasons. It costs millions to fix software, and your reputation is at stake."

Still, at the end of the day, "it's difficult to keep inadvertent vulnerabilities from being added to code," Davidson said. Oracle products have gone through security and assurance evaluations early and often, she said, with Common Criteria Evaluation Level 4 achieved for several versions of Oracle8. Oracle9i R2 is currently being tested.

But Common Criteria testing is no panacea for software quality and security, according to Shawn Hernan, team leader for vulnerability handling at the CERT Coordination Center, which is part of the SEI.

"Common Criteria doesn't do as good a job as it could to specify implementation quality," Hernan said. "Most security problems are from poor quality, and we see a lot of products with Common Criteria evaluation levels that have the same problems as noncertified products."

Testers and auditors have speculated that fewer security vulnerabilities arise from the highly rated software from Indian companies, Hernan said. However, CERT doesn't require software to be identified by country of origin, and there are no resources assigned to conduct such evaluations.

Offshore Buyer's Guide

Stories in this report:

Security

Bing’s AI chatbot came to work for me. I had to fire it.