Google's changes rely on guinea pig users

Google is using users as crash test dummies to measure exactly what changes it should make to its main search website — both to its famously Spartan search box and to the results it produces.

The method, called split A/B testing, involves Google showing different versions of the pages to users and measures how they respond, said Marissa Mayer, vice president of search products and user experience, in a speech at this week's Google I/O conference.

For example, Mayer said, the company wanted to find out how many search results to show users — the customary 10, or 20, 25, or 30? When asked directly, users said they'd like more results on a page, but testing showed otherwise.

Marissa Mayer shows three slightly different versions of Google's search results page that the company tested with users.

Specifically, Google found that when the results increased to 30 per page, people searched 20 percent less overall, Mayer said. After much analysis of server logs, the company found it was because it took about twice as long to display the longer results list for the user, and speed matters.

"As Google gets faster, people search more, and as it gets slower, people search less," she said.

The same effect happened with Google Maps. When the company trimmed the 120KB page size down by about 30 percent, the company started getting about 30 percent more map requests. "It was almost proportional. If you make a product faster, you get that back in terms of increased usage," she said.

Split A/B testing also led Google to refine exactly how much white space to pad around its logo and other elements on the search results page. And it changed from the industry practice of a pale blue background behind ads to a pale yellow background. People not only clicked on ads more, they also searched more in general, she said.

The subject clearly is close to Mayer's heart. She's an engineer who also has an interest in the more aesthetic realm of design.

"On the Web in general, [creating sites] is much more a design than an art," she said. "You can find small differences and mathematically learn which is right."

A history of Google's search page
Google's search page, with its abundance of empty white space and its almost boastful "I'm feeling lucky" button, looks downright ordinary today. But it wasn't always the case.

Mayer said that back when Google was a relatively unknown 80-person start-up, the company tested Stanford students on how well they could use Google to find which country won the most gold medals in the 1994 Olympics. The result: students would sit in front of the Google screen for 15 seconds, 30 seconds, 45 seconds, a minute...Google was perplexed.

So Mayer would eventually intervene and ask what was holding up the searchers. "I'm waiting for the rest of it," they'd say. Clearly they expected more of the flashy ads and busy text of other search pages of the 1990s.

"The very first home page was that misunderstood. People didn't resonate with it," Mayer said. One woman even thought the Web site was a fake construction that was part of a psychology experiment.

As a result, the company put a copyright notice at the bottom of the page. "It's not there for legal reasons," Mayer said. "It's there as punctuation. That's it. [It tells the searcher] 'Nothing else is coming; please start searching now.'"

Mayer oversaw much of Google's design, but the sparse start page wasn't her doing and wasn't even part of a plan, she said. Instead, it was the design of co-founder Sergey Brin.

Why so minimalist, she wondered? Sergey's response: "We didn't have a Webmaster, and I don't do HTML."

Google also decided against presenting newbie and expert versions of its search page, Mayer said. People figure it out quickly, so the company aims its product at the experts.

"The learning curve on search is really fast," she said. "People go from 'Where can I get spaghetti and meatballs in Silicon Valley' to 'italian food san jose' really fast," she said.

The complexity of search
Google tries to look simple from the outside, but its search process is, as no one will be surprised to hear, quite complicated.

A typical search will require actions from between 700 to 1,000 machines today, Mayer said.

That's grown more complicated as Google moved to what it calls universal search, in which the regular search results are mixed with results from its other search areas such as books, news, blogs, images, and maps.

With those other, narrower search services, Google lost sight of the simplicity users need in its haste to bring the services to market, Mayer said.

"The urgent can drown out the important," she said. "It's great we did these urgent, expedient search indices, but what we really need to do is put them on the same page."

Indeed, in the longer run, she envisions universal search growing far more sophisticated, with a page filled with "images, videos, and graphs--not a list of 10 URLs but as a holistic answer to your query."

Search also will become more personal, with results tailored for individuals — Google has begun offering personal results for those who sign up. One reason personalisation is important, she said: a very useful factor Google can weigh in its search results is what a person just tried searching for previously, she said. Knowing that, "We know what you discarded or are refining from," she said.

"We know 10 years out search will probably be a lot more personalised," she said. "And there will be a lot more content to index. When we think how to build search, it's important to think about the 10-year case."