What users say
Simon van Wyk, managing director of HotHouse, has years of experience in building and developing high-profile Web sites such as Toyota, HCF, i7's Olympics.com.au, and auction site Stuff.com.au.
In that time he's used various tools but concludes "[browser-based] solutions like Red Sheriff are the most flexible and the easiest to use...if I were a corporate [user], that's the way I would go." This approach puts an applet inside the page, so every access is measured.
"We've run Red Sheriff on a lot of sites and not had any problems with issues [such as cookies or JavaScript being disabled]," he says. With Red Sheriff "you can have a high degree of confidence in the number of first-time users versus returning users," for example.
The problem is that the service must set up the reports for you, and mapping the reports onto your needs can be difficult.
Forrester Research is another advocate of data collection through JavaScript or Java Web bugs, as it avoids proxy and browser caching issues. The firm recommends the use of content management tools to insert bugs on all pages, but warns that the site's privacy policy must reveal their use.
van Wyck doesn't think much of log file analysis. "There always seem to be problems," he says. If you lose one day's log, that throws the whole month's figures out, and you have to remember that month is no longer comparable to others. "I find log file reporting tools a damn nuisance," he says. "Most tools give you more data than you ever need--knowing which [data] to use is the trick."
Content management systems sometimes provide tracking tools, but "my experience is that they are somewhat over-engineered and don't work as well as advertised. They're difficult to get to work, and take a lot of maintenance," says van Wyk.
Hitwise is "dangerous" according to van Wyk, because he claims its Australian figures are skewed towards Melbourne, you don't get accurate figures that allow comparison, the numbers are based on impressions (so a site that makes visitors navigate through several pages to reach the information they want will return higher figures than one that provides more direct access), and you don't see the number of users.
Paul Strickland, Internet marketing consultant at Web developer Marketing For The Web has a more positive view of Hitwise. His company uses Hitwise figures when pitching for new business and when maintaining sites for its clients, which include the Accor Asia Pacific hotel chain, Delta Europcar, and Kea Designer Sportswear.
One of the main attractions is that Hitwise looks at enough traffic to allow a breakdown into detailed categories while restricting the figures to Australian users.
The ability to define your own categories is "very valuable," he says, as it means you can compare a site with those you regard as its competition rather than relying on someone else's categorisation. Strickland also spoke highly of Hitwise's ability to quickly implement requested features. "We find the service is very good," he says.
"We find the accuracy in rating visitation is . . . overall a good indication of what's happening on the sites," says Strickland. Marketing For The Web uses Hitwise figures in conjunction with the reports it receives from its hosting company.
For example, if the hosting reports show a site had 100 unique visitors per day and Hitwise says its share was 50 percent, you know the total market is only 200 visitors per day.
"We like to look at clickstreams," says Strickland. While a Hitwise report may show that a lot of visitors came from Hotmail, for example, he can distinguish users who opened a new browser window leaving Hotmail open in another from those who responded to links embedded within messages.
van Wyk also likes panel-based services like Nielsen//NetRatings because they produce figures that let him compare his sites with their competitors.
While panels might not be truly representative of the wider population, he doesn't think this really matters, as all sites within a category will probably be equally affected, so comparisons can still be made.
If you're operating a niche site that won't show up in a panel survey, "you're better off looking at your own data and traffic" rather than trying to compare it with others, he says.
It doesn't really matter if you have 50,000 or 51,000 visitors, he suggests, what's more important is knowing how many of those visitorsbuy from you and how many go on to your competitors' sites.
Clean, consistent data tells you about the trends even if the numbers aren't 100 percent right, he suggests.
Usability testing is important in van Wyk's book, but it must be kept in perspective. "Do little bits aimed at a specific outcome, and do it a lot," he suggests.
Research companies tend to turn testing into an end to itself, he says, and may confuse usability testing with opinion research. "I believe you have to be really careful about what you're testing, why you're testing it, and what you are going to do with the results. "Do as much research as you need to get an answer, then move on," he says. For example, if you get the same result from the first five people, stop testing.
"It's well worth spending the money on usability testing, no question at all," he says. Some of the results can be surprising. For example, small changes in wording can make a big difference to a site's success.
The Toyota site uses the phrase "Price Your Car" because that usability testing showed that wording better conveyed the function than other phrases. van Wyk has also found the word "buying" puts people off. "Looking," "browsing,"' "obligation-free," and the like appear to be less threatening.
