|
|
Following the news from my colleague Dr. Igor Muttik about his recent trip to Bilbao, Spain, to participate in the Anti-Virus Testing Workshop, AV-Test.org just released the results of their latest comparative test. It was picked up by many media outlets:
Unlike in many previous reviews, AV-Test.org ran various types of tests, and McAfee scored well in most of them:
| Test Type |
Rating | |
| Signature-based |
Good |
More than 90% detection rate out of 1 million files |
| Proactive |
Good |
|
| False-positive |
Very Good |
No false-positives detected out of 65,000 clean samples |
| Rootkits |
Good |
Detected all running rootkits except one |
| Response times |
Poor |
Around 6 to 8 hours |
Signature-Based Tests are usually an on-demand scan (ODS) by anti-malware products on a computer system against a set of known malware. We have discussed the challenges in making this test fair in the past.
Proactive Tests are similar to signature-based tests, except that they attempt to measure how well an anti-malware product can detect samples that it has never seen before–by taking an old DAT version and scanning with malware that was discovered after the DAT release date. This test often gives a sense of how well an anti-malware vendor does in writing generic, heuristic, or behavioral signatures. The caveat with this is that if a product ventures too far into this realm, the likelihood of false-positives increases.
False-Positive Tests are also an ODS test, except with a sample set of clean files instead of malicious files. False positives are the bane of the anti-malware industry as they could have far worst collateral damage than a false-negative (missed detection) depending on the severity. Because of our large customer base, we take this metric very seriously and have an internal zero-tolerance policy.
Rootkits Tests are one of the most complex and time-consuming tests that a tester can run, and are similar to the behavioral tests described above. However, these require even more intimate knowledge of both the target operating system and known rootkit techniques to accurately judge whether an anti-malware product was able to properly remediate the rootkit infection.
Response Times tests attempt to determine how quickly an anti-malware vendor responds to a new threat with their definition updates and heuristic detections.
Individually, each of these tests gives us a way to gauge one of the many facets of measuring the value of an anti-malware product. However, when grouped together, they can give a holistic picture of how well we balance the many criteria by which we are judged.
|
|
Congratulations on doing so well. Perhaps people would also like to look at a comparison of security products here – http://www.techsupportalert.com/review-security-guards.htm and comment
Props to you guys for publishing your results on your own blog even with the ding for response times. Personally I think a 6-8 hour response time is not bad at all. I assume it meant 6-8 hours after it is confirmed malware by either McAfee or other industry peers. With the volume of malware that has to be analyzed/processed/detected I say not too shabby gents, not too shabby at all.
Also, too bad the opportunity to snipe ole CMantis in COD4 (previously 1,2) has diminished, I believe the new version would be a good test of our keyboard kung fu.
Good work guys!
Abhishek be quiet.
Submit your own comments / message for this post