Testing Google's Panda algorithm

analysis Google's sweeping changes to website rankings have roiled the web industry, including the company's announcement last week that its algorithms now incorporate more "user feedback signals".

Google visualised: top websites like Wikipedia, Amazon.com, YouTube and IMDB are red and orange. Less popular sites, in blue and violet, were less visible on Google's first page. (Credit: Declan McCullagh/CNET)

The reason why Google made such a dramatic change to how it ranks websites is simple: search engine optimisers had learned how to game the earlier algorithm to make low-quality writing more visible than quality content. Instead of preparing web pages designed to benefit readers, SEO-focused content farms were writing for search engines.

To test the changes and provide a rare glimpse into Google's algorithmic workings, ZDNet Australia's sister site CNET compiled nearly 100,000 results by testing Google.com in March, and again last Friday after the most recent alterations took effect.

News sites generally benefited from the changes. According to our rankings based on the number of appearances on the first page of Google results, Fox News moved up from the number 89 spot to number 23. ABC News had a similarly impressive uptick, and ESPN, The New York Times, and Yahoo News became more visible as well.

The "Panda" algorithm change dramatically lowered traffic to sites like AssociatedContent.com, FindArticles.com and EZineArticles.com, according to a post by SearchMetrics.com. It also negatively impacted some perfectly legitimate sites, including Cult of Mac and the British Medical Journal.

CNET's analysis found no significant change among the very top sites, which remained the same. Wikipedia, YouTube, Amazon.com and IMDB stayed in the same enviable tier one positions, respectively. Hulu.com surged to position number22 from number 51.

Twitter, Facebook and Huffington Post each moved up a single notch, with Yelp, Flickr, Apple.com and WebMD slipping a bit. US government websites received a boost, with WhiteHouse.gov climbing from number 125 to number 79, and NASA, the Center for Disease Control and the National Institutes of Health increasing as well.

Among the websites that slid in visibility: WikiHow and eHow, which is consistent with other reports that Panda lowered the ranking of so-called content farms. The comparison site Nextag.com also slid.

"People who got hit were trying everything to get their sites out of it," says Barry Schwartz, news editor of Search Engine Land. "It was targeting low quality content sites."

Google declined to elaborate. "We typically don't comment on how specific algorithmic improvements impact specific websites," the company said.

How we did this

To generate these results, we compiled approximately 2000 search terms from a sampling of Google Insights' web, news and shopping searches. We then removed the duplicates, resulting in a total of 1656 search terms, and tested those with Google.com (while not logged in) to see what the results would be.

We ignored advertisements, Google shopping results and "searches related to" suggestions. We did decide to include Google News results, even though they're relatively ephemeral and can change by the hour. Plus, our analysis showed that excluding them wouldn't have changed the results very much.

Now, the disclaimers: Google, as it will be the first to tell you, is constantly altering its algorithm, and by the time you read this, the results from Friday's searches could well be out of date. Our first scan was in March, after Panda's appearance in late February, so it likely didn't capture the most significant changes.

Also, this shouldn't be viewed as a representative cross-section of web searches. Google Insights only includes the most popular requests, not the more obscure ones. It focuses disproportionately on current events and — because we borrowed terms from the shopping searches — products, especially tech gadgets.

Then again, "charlie sheen teeth" and "venereal disease" appeared in our list of search terms. Thank you, Google Insights!

Google's localisation algorithm

We also tested what happens if you connect to Google.com from an overseas internet address. We picked one in London. We performed the same searches on the same day — the only variable that should have changed, in other words, was our location.

The results? Google engages in significant localisation efforts, as you might imagine, with Yelp.com being the largest beneficiary by far.

In searches originating from the UK, Yelp appeared only twice. In US searches, by contrast, it was the ninth-most popular website, with both its topic and individual business pages weaved seamlessly into the main search results.

From our California address, Yelp garnered an enviable 45 first-page appearances for generic searches like "chocolate", "cleaning", "food," "lights," "laundry," "tv" and "weddings."

Other big localisation beneficiaries that appeared prominently in US searches but not from the UK: Davidsbridal.com, BarnesandNoble.com and Walgreens.com.

In addition to highlighting nearby bookstores and drugstores operated by national chains, Google also heavily favours local businesses.

For our US tests, we used an internet address near Palo Alto, California, which prompted Google to rank nearby businesses and municipal websites near the top of search results.

The City of Palo Alto's website appears in the first page of search results for terms including "adventures", "art", "business", "gas" and "jobs". PaloAltoOnline.com makes repeat appearances ("budget cuts", "restaurants"), as do Stanford, the Palo Alto Medical Foundation and Mike's Bikes.

There's not as much localisation in the other direction. But the BBC's website leaps from the number 66 spot to number five, and the UK's National Health Service (which made no appearance in the US) shows up at number 26. The visibility of Amazon.co.uk, the UK pharmacy chain Boots and NetDoctor.co.uk also jumps dramatically.

We wondered if connections to Google.com last month from abroad bypassed Panda and used the earlier algorithm, which would have made for another intriguing test. But an informed source close to the company, alas, says that's not the case.

See for yourself

Below you'll find an Excel file with multiple spreadsheets containing the raw data. If you use the data for any purpose, please attribute it to CNET and include a link to this article.

Four of the spreadsheets (March US, April US, March UK, April UK) should be self-explanatory. The others show comparisons and may require a bit of explanation: the first column is the hostname, and the second and third columns show how the ranking has changed from the point of comparison. The final columns represent the search terms that bring up that website on the first page of Google.com results.

In the case of the UK spreadsheet comparing March to April, the second and third columns indicate that Facebook.com moved from number 11 to number seven. The difference is four, which shows a positive change (negative numbers are the opposite). For the "April US versus UK" spreadsheet, those columns show that Yelp moved from rank 1328 to an enviable rank of nine because it benefited from Google's localisation efforts in the United States.

"NA" means that the website didn't exist in the spreadsheet being used for comparison. In the "April US vs UK" spreadsheet, "NA" shows up because the UK's National Health Service website doesn't appear in any US searches for the terms we tested.

If you find anything interesting, or have any suggestions, please contribute to the discussion below.

Files:

Excerpts on Google Docs (limited because Google Docs allows only 400,000 cells)

Full spreadsheet in Excel format (.xlsx.gz)

Disclosure: McCullagh is married to a Google employee who is not involved with Panda.

Via CNET