IDG News Service - A coalition of security companies and researchers has agreed on guidelines for how security software products should be tested, which may help put an end to long-running disputes about different testing methodologies.
Two sets of guidelines covering principles for testing security software for performance and testing entire security suites were adopted by the Anti-Malware Testing Standards Organization (AMTSO) at its latest meeting in Helsinki.
The guidelines are part of a series of documents on AMTSO's Web site aimed at introducing a set of standards broadly agreed through the industry as appropriate for ranking the effectiveness of security software.
Security companies have bickered over tests run by organizations such as AV-Test.org and Virus Bulletin and others, which regularly test and rank security software.
Among the contentious issues are the age of the malicious software samples used to test antivirus programs. Companies that failed to detect certain kinds of malware have argued in the past that the samples weren't threats any more and that they had removed the signatures from their databases.
Other companies have countered that their failed test wasn't accurate since other technologies in their software would have stopped the particular threat. Some antivirus testing programs perform static testing, where an antivirus engine is run against a set of samples.
But many security companies have incorporated other complex ways to detect malware. One such method, known as behavioral detection, checks to see how malware behaves on a computer.
The new guidelines are voluntary, but many testing organizations have agreed to go along with them, such as Virus Bulletin, AV-Comparatives, West Coast Labs and ICSA Labs, said David Harley, an AMTSO board member and director of malware intelligence for ESET. AV-Test.org will also use the guidelines, according to Andreas Marx, who founded the company.
"We're just trying to get people to think harder their methodologies so that they actually make sense," Harley said. "It doesn't mean you can't do things different ways, it just means you have to try and conform to a rationality."
Virus Bulletin has contributed to the guidelines covering performance testing, said John Hawes, technical consultant and test team director.
"We've already started implementing some of the ideas developed while discussing and designing this document, with some major expansions to the performance data we report in our comparatives in recent months and more improvements on the way," Hawes said. "We're also hard at work developing a new style of test which will allow us to measure the full range of features in many of today's security solutions."
The guidelines may not entirely end the feuding between security companies and testing organizations but will better inform those who write reviews of security software, said Stuart Taylor, manager of Sophos' threat lab and chairman of the board for AMTSO.
- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- The 12 PCI DSS 3.0 requirements addressed by Peer 1 Hosting This handy quick reference outlines the 12 PCI DSS 3.0 requirements, who needs to be compliant and how Alert Logic solutions address the...
- Defense Throughout the Vulnerability Life Cycle This whitepaper provides insight into how to leverage threat and log management technologies to protect your IT assets throughout their vulnerability life cycle.
- Mobile Policy Checklist Here's what to consider when putting together a mobile policy designed to support a highly productive workforce.
- Securing BYOD Mobile computing is becoming so ubiquitous that people no longer bat an eye seeing someone working two devices simultaneously. Individuals and organizations are...
- Live Webcast On-demand webinar: "Mobility Mayhem: Balancing BYOD with Enterprise Security" Check out this on-demand webinar to hear Sophos senior security expert John Shier deep dive into how BYOD impacts your enterprise security strategy...
- Live Webcast Endpoint Backup & Restore: Protect Everyone, Everywhere Arek Sokol from the bleeding-edge IT team at Genentech/Roche explains how he leverages cross-platform enterprise endpoint backup in the public cloud as part...
- Streamline Software Asset Management, Compose a software Management Symphony Keeping track of your organization's software is easy with effective software management solutions from CDW. View the videos in our software solutions channel
- Druva inSync: Endpoint Data Protection & Governance CLICK HERE to watch this video about protecting corporate data on laptops and mobile devices, sponsored by Druva. All Security White Papers | Webcasts