As the CEO of a Group Buying business in the nascent and burgeoning category it was critical that I had a very clear view of marketing effectiveness, with Audience Engagement being the key indicator. There were a number of sources available to the team that purported to provide reliable Audience measurement and insights however I only depended on two to provide an accurate view, Omniture and Nielsen.
Alternative sources included Hitwise, Google Analytics and Alexa – Google Analytics is cheap/free but pretty unreliable and Alexa provides a Relative view only. Hitwise is the worst of the bunch though given their data collection methodology means it doesn’t represent the broader online population and worse still, it doesn’t necessarily reflect human activity!
Here are the two main issues with Hitwise data:
1. Hitwise does not measure individuals – it measures traffic.
This effectively means you could hit your website with bot traffic to boost your numbers and it would show as traffic in Hitwise. Nielsen Australia removed 50% of GroupOn Australia’s traffic in March because that traffic consisted largely of unsolicited clicks, meaning popups that appear as you close scurrilous ads (Congratulations, you have won $1,000,000!!!!!) – those clicks are still counted in Hitwise.
2. Hitwise doesn’t include key ISPs
Hitwise harvest data from partnering ISP’s, however Australia’s two largest ISP’s BigPond and Optus don’t participate. This is major a concern as a large proportion of internet users (about 58%) are not reflected in their data. This is a particular problem for a business like Cudo given its mainstream audience, and mainstream Australia do not typically use fringe ISPs.
Nielsen was recently selected as the official measurement partner of the Australian IAB, in their press release they said:
With the endorsement of Nielsen Online Ratings, IAB Australia is identifying people-based metrics, as opposed to browser-based, as the best and preferred online audience measurement system for the Australian online advertising industry.
This is the nub of the problem. TechCrunch called it out almost two years ago.
At Cudo we didn’t care about browsers for obvious reasons, we cared about people, they still do, like the 1,000,000 plus Australians who go to cudo.com.au each and every month, I couldn’t give a monkeys how many Bots swing by!
57 Signals 
Agree on Hitwise – fun to look at and can give good insight into what competitors are doing well or badly but a joke for measuring market share.
Disagree with “audience engagement” being your key metric, it’s a lot simpler with transaction volume and revenue being all that matters.
I don’t like Nielsen’s new hybrid people approach. Giving an unknown multiple to a small panel of people is not much better or representative than Hitwise.
Google Analytics set up well (with AdWords and ecommerce tracking) is going to provide a lot better view of online marketing effectiveness than either Omniture or Nielsen.
Of course when you’re trying to connect something inherently disconnected (how many people visited my website sometime in the future after seeing it on TV?) any tool you use is going to be hit and miss.
Nielsen is also flawed since you have to pay to be measured. This misses out on the vast majority of websites that Australian people visit. So, I would say that the Nielsen methodology is flawed since it doesn’t capture all “human behaviour”.
Also, Hitwise actually does report on Visitors that seems to correlate with our Omniture data, so I’m not sure if the bot problem you mentioned is as big as what you suggest – however, it is worth noting in the methodology when using the data.
Nielsen also suffers from implementation issues. Due to the fact that the code has to be placed voluntarily by the company being tracked, there is significant room for inconsistency in the implementation across sites – are subdomains tracked?, are tags being fired once per page?
Furthermore, Hitwise gives far more information than just visits and time on site. Nielsen does do a great job at measuring visitor cross domain behaviour, but this is the only advantage over the Hitwise methodology which has clickstream data, ranking data, search term pathing, website pathing, search term data (paid/ organic), among other things – and covers the entire web (for the 3 million users in the sample – which most statisticians would argue to be statistically significant in a population of 21 million).
There is a lot of points I would like to touch on but I’ll keep it to a minimum. Firstly, im in complete agreement with you about Hitwise and many other poor excuses for analytic tools. There are much more comprehensive measurements now.
@Matt – I disagree that Google analytic has legs. It isnt as easy as people think to set up and it is still not the best for attaining organic measurements of you visitors. There are better tools that have good measurement tools out there including Ominture.
On CMSwire I found Sitecore. It is the best I have seen by far for this kind analytics. I watched a webinar last week by Darren Guranncia (spelling?) product manager. It showed off some of the best measurement techniques and orchestrated website marketing I have seen in a CMS or analytic tool. If you want organic measurements, not bots, not just visits but how user engagement data this was my pick. The webinar was called stop the random acts of marketing. There might be a recording floating around
Thanks Nick, I’ll check it out.
Cheers Nick, I’ll head over and check it out as well.