To catch a spy: Anti-spyware tools reviewed

Test results
Clean machine accuracy and performance testing
  • Accuracy: Only Lavasoft and Spybot Search & Destroy picked up anything when instructed to scan a newly installed and patched version of Microsoft's Windows 2000 Professional. Both reported Alexa (adware) related items. The other seven applications in this test correctly reported no items.
  • Performance: Lavasoft was the fastest followed by Computer Associates and then Symantec (for its quick scan).

Infected machine accuracy and performance testing
Out of the six central management capable enterprise anti-spyware applications in this review (McAfee, Symantec, Computer Associates, Trend Micro, Lavasoft, and Webroot) Computer Associates was the easiest application to use and the quickest to scan (45 seconds). Its detection rate came equal second with PC Tools (six detected).

The biggest disappointment of the testing was Lavasoft, which only managed to pick up one out of the nine items. The second last, surprisingly, was Trend Micro with three detections only. Microsoft, Webroot, and McAfee all came in with four detections each.

Symantec came out on top but only when running its full-system scan. It hit nine out of 10 of the installed spyware and adware programs. Its quick-scan performance was disappointing, coming in at just four detections. The problem with the Symantec full system scan was that it took more than 40 minutes to complete. Bear in mind, however, that both Symantec and McAfee are scanning for viruses as well as spyware so one would expect slightly more time to be taken. That said, 30 (McAfee) or 40 minutes (Symantec) is still a big leap from the next slowest -- Webroot at a shade under nine minutes. All other recorded times were less than seven minutes, with Computer Associates the quickest at just 45 seconds.

Summary:
Clean performance

1. Lavasoft (smart system scan)
2. Computer Associates
3. Symantec (quick scan)

Infected performance

1. Computer Associates
2. Lavasoft (smart system scan)
3. Trend Micro

Accuracy in detection (out of 10)

1. Symantec (full scan) 9
2. Computer Associates 6
3. PC Tools (quick and full scans) 6 each

< Prev 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Next >

Like this article? Click below to send it to your mobile for free!

Advertisement

Print feature brought to you by Hewlett Packard

Talkback 34 comments

  1. Such an accurate review Roger LeBroy -- 05/12/05

    Good to see effort was put into actually testing these products. Great Work !

  2. Yes but... Anonymous -- 06/12/05

    While SCS works great on our 300+ user network, SCS 3.0 is very buggy. It often locked up PC's with memory errors, usually during email scanning. If you get SCS, make sure you get 3.01 with the latest patch applied. 3.01 is very stable.

    1. Mac Anonymous -- 06/12/05

      The best thing about SCS is its updateability. New detections are constantly being added via virus updates. It’s great that no additional product patches were needed to gain more functionality.

  3. Spybot Anonymous -- 06/12/05

    Spybot - Search & Destroy was listed in the article, but the test results were not included except to point out that it did incorrectly find Alexa. Why is finding Alexa incorrect? Is spyware not spyware if it is included in a Windows install?

  4. Spybot Anonymous -- 06/12/05

    Spybot - Search & Destroy was listed in the article, but the test results were not included except to point out that it did incorrectly find Alexa. Why is finding Alexa incorrect? Is spyware not spyware if it is included in a Windows install?

    1. Check the definition of spyware. Spybot is right. JoiseyBill -- 06/12/05

      If you check Secunia advisory SA8955,
      http://secunia.com/advisories/8955/
      or the page referenced from there on imilly.com
      http://www.imilly.com/alexa.htm#subvert
      ...
      Or just google Alexa + spyware ...

      You may agree, as I do that Spybot's detection is correct. The other products are wrong.

      Further, your testing method doesn't mention whether you use the "out-of-the-box" configuration, or if you actually tweak the alarm/ignore lists. I know that MS AntiSpyware and Spybot each give the user some tools to toggle the "spyware-ness" of pre-defined items.

      This is useful, say if you install VNC or similar remote tools - you want to turn off the alarm when your program detects this.

      If your test parameters say that Alexa is not spyware, then you have a problem with either the supplied definitions or the default configuration. Please make your determinations a little more precise.

  5. EULA are worthless Anonymous -- 06/12/05

    First, IANAL!

    What happens when my children install software? They click right through the EULA and it means nothing! They are minors! They can't enter a legally binding contract. Have you ever seen a EULA that CLEARLY says right at the top "YOU MUST BE OVER 18 TO INSTALL THIS"? I haven't and until I do I will continue to maintain the EULAs are worthless!

    1. EULAs and children Anonymous -- 15/12/05

      > What happens when my children
      > install software? They click
      > right through the EULA and it
      > means nothing! They are minors!

      Then why do you let them have administrator rights on the PC?

      If they can't understand the ramifications of what they're installing, don't let them do it!

      > They can't enter a legally
      > binding contract. Have you ever
      > seen a EULA that CLEARLY says
      > right at the top "YOU MUST BE
      > OVER 18 TO INSTALL THIS"?

      I've never seen an EULA that clearly says anything. Their entire purpose is to be obfuscatory and arse-covering.

      > I haven't and until I do I
      > will continue to maintain the
      > EULAs are worthless!

  6. Doesn't reflect real world results! Anonymous -- 06/12/05

    If you have over 20 users you will want centrally managed capability. So you can eliminate PC Tools, Microsoft & Spybot S&D. I have used CA, Symantec & Webroot. By far the best was Webroot. I can't believe CA & Symantec's products beat any of the applications in this shootout! Hands on experience has shown me that both products have poor detection and removal rates. CA's eTrust was the worst application I've ever used. Symantec's product is only slightly better with it's detection of spy/adware, but it's removal success rate is extremely poor. Webroot has consistently sustained superior dectection/removal success rates. This article is misleading and is a poor source for antispyware application comparisons. Please do not use this article to make a purchasing decision. There are many other reviews on the internet that give a more accurate representation of each applications strengths and weaknesses.

    1. Or if they don't agree to your ideas. Anonymous -- 09/12/05

      I'm using CA's eTrust Internet Security Suite. This has Anti Virus (actually rebranded VET Antivirus) Firewall (Zone Alarm pro rebranded) and PestPatrol. This suite was very cheap. I got it for $30 US off the web.
      I have been using these products from before CA bought them and have found them, overall to be the best. VET has always been a brilliant antivirus (fast accurate and a small memory footprint) Zone Alarm pro - good Firewall. PestPatrol - great, also has good web page for extra info.
      I have used others, including Norton, Adaware, Spybot, etc. Good products (well maybe not Norton) but CA's package of AV, Firewall, antiSPAM and PestPatrol is superior.
      Overall the best protection I have come across.

  7. EWIDO owns u Anonymous -- 06/12/05

    I checked out Ewido after a recommendation from a microsoft employee (they didn't recommend their own)

    it's great. check it out ewido.net

    it found 450 items in addition to the 45 that spybot found

    1. I agree Anonymous -- 13/12/05

      Ewido security suite is by far the best anti-spyware/malware removal tool I have found for single user use. Enterprise editions are great, but only if you're an IT manager with 10 or more computers being managed. I can't believe ZDnet didn't review Ewido. Bad form.

  8. Sorry, is ZDNet faking reviews now? Charles Merriam -- 06/12/05

    It just seems odd the SpyBot Search and Destroy, which appears to be the most used program, did not have published results. It should have ranked high for individuals and terrible for companies.

    I've been finding issues with other ZDNet reviews as well. Does anyone have a different review of SpyWare detectors?

    1. Reply from the Engineer who performed the review Matt Tett -- 06/12/05

      Hi There,

      Thanks for all your comments regarding S&D, I did actually perform the exactly the same testing on S&D as with the other products and submitted the results to ZDNet with the review, they obviously have misplaced them or unintentionally left them out.

      Hopefully following these comments from readers they will post the S&D results online.

      Thanks again for your feedback.

      Regards,

      Matt Tett

    2. RE: Reply from the Engineer... JoiseyBill -- 07/12/05

      Thanks Matt, for sharing your work and for doing a very thorough job.

      I've already noted my single (relatively minor) disagreement, but I also wanted to show support.

      As far as the general negatives others have thrown out -
      *other ZD magazines post reviews with other results. Just because somebody's favorite software didn't make this review doesn't imply a bias.
      *this article focused on a finite set of "entrprise class" software. There are hundreds of packages out there. This study looked a few of the more popular.
      Maybe the other companies didn't respond to the reviewer's request, maybe the software isn't truly enterprise class. How much time & resources do you think should be put into one study for one article - when we know the results will be all but worthless in six months?

      I think this was a good piece of work. I'm glad to see that people are reading it critically and discussing it. This is an important topic.

    3. PC Magazine Does the most thorough reviews - Webroot is Editor's Choice John Lavelle -- 09/12/05

      Testing spyware products against 9 peices of spyware is grossly negligent given the amount of spyware in the wild with over 3,000 active spies. For a real review look at PC Magaizine. Much different test methodology(they actually have one) and much different results.

      http://www.pcmag.com/article2/0,1895,1879983,00.asp

    4. spyware John Taylor -- 22/12/05

      Soryy old chap but you miss the point here, what ZDNet were testing is a Corporate solution, centrally managed, and ideally integrating into an overall security policy. That is very different from a stand alone product. Incidentally there are over 30,000 spies not 3,000.

  9. once again, u dis free software Anonymous -- 06/12/05

    hello and thanks for reading this,
    there are many free products that you do not review as it seems is your ongoing policy

  10. AOL Spyware Protection utilizes Pest Patrol Joe M. -- 06/12/05

    Just like to add that AOL Spyware Protection 2.0 is based on Pest Patrol; and is free for AOL Members

  11. Typical to leave out free products tony -- 06/12/05

    I find in passing strange that the free product SpyBot was not included. The only conclusion I can come to, is the usual corporate bias against free products.

    I work in an medium size organisation which uses Spybot S&D very successfully on more than 300 PCs. It can be rolled out and updates easily enough (yes I know it doesn't have central management capabilities, but if you're using Windows servers and associated management tools, it's not hard).

    1. Spybot was included Matt Tett -- 06/12/05

      Read my post above, SpyBot was included, the results were not published for some reason.

      Also if you could publish your names and occupation details, instead of hiding behind Anonymous please ?

  12. Which items of spyware were used? Roger -- 06/12/05

    Various vendors dispute the findings of this report, I suppose they would, but your story would be more complete if you detailed exactly which spyware items you installed, and which were detected and which were missed for each product. With only 9 items, this would have been a pretty straightforward table, the fact that you have left this information out leaves your findings open to criticism.

  13. Because it's not actually there... Anonymous -- 06/12/05

    Finding Alexa is obviously incorrect if Alexa is not present on the test system...

    1. RE: Because it's not actually there JoiseyBill -- 07/12/05

      "...Finding Alexa is obviously incorrect if Alexa is not present on the test system..."

      Fact:
      Secunia and others [see my previous post] point out that there is a facility in Internet Explorer that reports information to Microsoft and Alexa.
      No one has disputed this.

      Some websites say Microsoft has released a "patch" for this issue (they hide the button, but don't change the code) for post XP-SP2 versions of IE.
      * But this test used Windows 2000, not XP.* Therefore, this "fix" is moot.

      Fact: Microsoft appears to have re-evaluated their position on this, and are now providing full privacy disclosure with Windows Longhorn betas:
      http://www.microsoft.com/windowsvista/privacy/ieprivacy_pr6.mspx

      From that site: "...If you do not wish to send the address of the Web page you are currently viewing to Alexa, do not click Show Related Links"
      Again, this disclosure is not included with Windows 2000.

      I'm not sure what definitions you are working with, but code that sends out information about me - without my consent, and especially to 3rd parties who I have never done business with - IS spyware.

      Therefore, Ad-Aware , Spybot S&D and any other program that identifies this is correct.

      Perhaps there is some confusion in this issue becasue there is also a second "Alexa spyware" that installs the Alexa toolbar. This other spyware is identified by Symantec & others.
      http://securityresponse.symantec.com/avcenter/venc/data/trackware.alexa.html

      I hold and affirm that these detections are accurate, and not "false-positives".

      As another reader succinctly commented <paraphrased>:
      "just because Microsoft installed it, doesn't mean it isn't spyware"

  14. Methods for determining accuracy of removal David Bowser -- 07/12/05

    I did not see any mention of logging methods for infection. With a new system, there must be a method to determine what will change during the infection, otherwise, there is no way to measure the effectiveness of the detection and removal processes. A program might tell you it found and removed 500 malware fingerprints on your system, but if the changelog indicates the malware created 1000, then the tool is not very effective.

    There is also the matter of how to count malware fingerprints. If one program counts a directory as a fingerprint, but not the files within, whereas another counts the directory and the files, is the latter more effective? What if they both remove the same directory and the files within?

    FileMon and RegMon are pretty good tools for this type of analysis.

    http://www.sysinternals.com/Utilities/Filemon.html

    http://www.sysinternals.com/Utilities/Regmon.html

  15. What About CounterSpy? Nic van Zant -- 07/12/05

    You missed one of the most important products. CounterSpy. Why???

  16. Search & Destroy is not for corporate use Anonymous -- 07/12/05

    The EULA for Spybot specifies that it is not for corporate use. This is overlooked consistently in articles and reviews.

    1. Spybot Corp use Anonymous -- 08/12/05

      You can use S&D on corporate networks. That would be why they have an enterprise server....
      -Nate

  17. Your review is dishonest. Anonymous -- 07/12/05

    This review leaves out the best product: ZoneAlarm Security Suite, which handles all issues: Spyware, Viruses, and Firewall.

    Fake reviews which avoid considering the best product are becoming quite common.

  18. Gallery of morons vealmince -- 09/12/05

    How many readers have posted idiotic comments like "This review is inaccurate/dishonest/biased/worthless because it doesn't include [product that I like] or because its results don't tally with my experience with [product that I like] in my very specific set of circumstances"? Are you people completely incapable of recognising that an entire world exists outside your puny little minds?

  19. Where's Spybot SD results? Anonymous -- 10/12/05

    Good review, but I really missed the Spybot Search and Destroy from result table.

    Why isn't there? What are results on cleaning abilities?

    Also, why not a Price/Performance evaluation? This is the most important for domestic user...

  20. Do ZDNet get comission from Symantec? Mike Caddick -- 14/12/05

    Seems that whenever there is a security, anti-virus or anti-spyware roundup, the guys here at ZD Net always give the Symantec products very high scores.
    Perhaps they've never actually used these resource hogging, crash prone bloatware in the real world.
    On EVERY single installation of Symantec Anti Virus I've encountered (and I encounter several per week) a quick scan using one of the free anti-virus offerings like AVG or Avast ALWAYS turns up infections that NAV has missed.
    They seem content to trade on their previously good name and advertise the heck out of their products instead of actually creating good software with a small footprint that doesn't bring the whole system crashing to its knees.
    At least the more recent versions can be uninstalled without reinstalling the entire OS like you needed to when trying to completely uninstall NAV 2003.
    Go into just about any forum about security and protection software and you'll see a litany of problems that those unlucky enough to use symantec programs come across day in day out.

    1. As if vealmince -- 16/12/05

      Mike, I have used Symantec AV here for years and it works fine - never had a problem. Therefore you are wrong and stupid and ignorant and your opinion is worthless. Are you paid by McAfee to say bad things about Symantec? I bet you are.

  21. Spyware Firewall Tony G -- 29/07/06

    Not mentioned, freeware http://www.ecommsec.com

Add your opinion


Back to top

Featured