A slew of security companies have banded together to tackle what they claim are falling standards in the testing of anti-malware applications.
The Anti-Malware Testing Standards Organisation (AMTSO) was formed on Monday by 25 companies with an interest in IT security, including Microsoft, Symantec, AVG, McAfee and Trend Micro.
While organisations such as Virus Bulletin already exist for the testing of anti-malware products, the formation of AMTSO was "driven by industry-wide concern about the increasing mismatch between what anti-malware technologies actually do and the testing methodologies used to evaluate them", according to AMTSO.
Most current evaluation methodologies for anti-malware products are inadequate, according to a statement by AMTSO: "As anti-malware solutions become more complex, many existing tests are unable to evaluate product effectiveness properly."
Security vendor Sophos, which belongs to the organisation, agreed that testing standards had not kept pace with the evolution of anti-malware products. "Traditional tests are becoming increasingly irrelevant as they fail to take into account the new technologies built into security solutions," said Stuart Taylor, SophosLabs manager.
While AMTSO has not mentioned any specifics on its testing agenda yet, head of technology at Sophos Australia Paul Ducklin believes that the new body will focus on a number of areas hitherto neglected in testing anti-malware.
"One of the biggest problems the industry and consumers have had with antivirus software testing is the over-reliance on increasingly meaningless percentage figures rating the efficacy of a product," said Ducklin.
"To really put an antivirus up to the test you need the same resources and expertise that an antivirus writer has in the lab, and the sites and organisations that generally conduct the testing just don't have access to that," he said.
According to Ducklin, many products are currently only tested against the "wild list" -- a register of current and active malware threats -- when what is really needed is the ability to test against completely new threats and some others long forgotten.
Ducklin added that by establishing AMTSO as an independent industry body, resources will now be available to test new anti-malware without favouring one vendor or another.
The organisation's aims include providing a forum for discussions related to the testing of anti-malware products, and developing and publicising standards and best practices for such tests.
