« May 2007 | Main | July 2007 »

June 21, 2007

More on the Comscore cookie deletion study

fdcookies A bit late to the punch with this one (as ever), but Comscore have published a white paper which contains more detail about the methodology behind their recent study which showed that both first and third-party cookies are deleted more than we thought (or at least more than we hoped). It clears a few things up about their claims; and raises some interesting new questions.

Log-in vs 'passive' cookies
The Comscore study used 'passive' (non-login) cookies from Yahoo! for its first-party cookie. As per my previous post on this topic, the high deletion rate of this cookie (31% at least once a month, according to Comscore) reiterates the need to use login information (i.e. cookies) for UU counting where possible.

Cookie 'flip-flop' vs cookie deletion
Buried in the numbers on page 8 of the report is a footnote about 'preserved' cookies. According to Comscore:

"Preserved designation includes PCs where two or more distinct cookie values were observed alternating throughout the observation period. Such oscillating patterns reflect the use of multiple browsers, or multiple accounts on a PC, and do not reflect reset events."

From the study data, 69.3% of first-party cookies were either constant for the month of the study, or 'preserved' according to the above definition. Another 16.1% were reset once (meaning that there were two distinct cookie values during the period). But in order to tell whether a cookie value change is due to deletion or multiple accounts, you have to have at least three cookie values: A, B, A. I'm going to call this a 'recurring' cookie value. So there's a good chance that a proportion of the "1 reset" group is actually recurring cookies where there was only an A, B pattern (i.e. the recurring cookie didn't come back again before the month was up).

The report isn't clear about whether the study stripped out recurring cookies from the reset counts in higher groups. For example, if they saw the following cookie values through the month:


then that is clearly three resets. But if they saw the following values:


then is that three resets, or did they strip out the extra A and call it two resets?

This is important because one of the banner headlines from the study is that 7.1% of computers contribute 36.3% of the unique cookie values, resulting in an average overstatement of UUs of 150%. If actually these numbers haven't had recurring cookies stripped out, then the overstatement wouldn't be that high.

Cookie awareness
One interesting aspect of the study which wasn't in the original press release was the results of some survey questions that Comscore asked. The most interesting one was "Do you know the difference between a first-party and third-party cookie?" An astonishing 29.8% said they did - I'm not sure even that many people here know the difference. Only 4.2% claimed to selectively delete third-party (but not first-party) cookies, however, which is unsurprising (or surprisingly high), since it's basically impossible to distinguish between first- and third-party cookies once they're on your system, unless you keep a database of the sites which are known to issue third-party cookies.


Overall, the study makes for interesting reading, and seems to have been undertaken with some care. However, towards the end Comscore throws in a bunch of other reasons (rotating IP address, accidentally including international numbers when comparing with domestic panel data) which also inflate server-based counting methods. The addition of this extra material simply has the effect (for me, at least) of making the report seem even more self-serving - it seems disingenuous to release a paper (written in however scholarly a fashion) which basically just bashes server-side measurement when Comscore's motives are so easy to see. It contributes to the debate, and I welcome it, but it leaves a nasty taste in my mouth. Not something that can be said for the splendid Father's Day gift that I received on Sunday (my family know me too well).

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon

June 19, 2007

Hand-crafted data visualization

If you're a big brand advertiser and you need to get your high-up marketing execs excited about web analytics, what can you do?

Well, one thing you can do is pay someone to build you a really funky interface for your web analytics data:


Much in the vein of 3D Site Stats, this type of 'analytics' is designed more for entertainment/engagement than serious analytics. It's created by German agency Scholtz & Volkmer, and is up for an award at this year's Cannes Lions festival. It has some quite nice visual features (like highlighting a particular data line when you hover over the 'person' representing a part of the site or audience), but its best feature is only revealed when you press (and hold, for about 20 seconds) the D key on the keyboard. Heh.

Funnily enough, Webtrends hired Coke's former interactive marketing VP, Tim Kopp, as their VP of marketing in January. I wonder if Tim had anything to do with this project?

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon

June 14, 2007

Dear John...

john-marshall I know we've had some rocky times together, but you can't just walk out on me like this. Was it something I said? I know that at times I disapproved of your relationship with Avinash, but with a relationship as long-standing as ours, a little jealousy's going to set in eventually.

So where will you go now? Will I see you growing your hair out and hanging around in bars with Eric? Or shaving your head and pontificating with Jim? Don't tell me you have a new flame already - or are you going back to Marc? I know you always had a soft spot for him.

Well, wherever you end up going, good luck, I guess. There'll always be a warm welcome for you here, when you're done flirting with your fancy new friends.

Yours bitterly,

The Web Analytics Industry


[PS Seriously, good luck with your move, John. You've done a great job at ClickTracks and I for one will be watching carefully to see where you apply your talents next]

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon

June 04, 2007

Bring the love back

I'm a sucker for a nice little TV short, and this amused me at the same time as giving me that glow you get when the behemoth of a company you work for does something cool:

My favorite line from this is when the guy says, "Know you? Sweetheart, I know everything about you! You're 28... to 34".

To learn more about the film and the way it's being used, visit www.bringtheloveback.com.

[Update 6/5/07: Thanks to Kip for pointing out that I had the URL above totally wrong; I've corrected it now]

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon

June 01, 2007

The miserable business of making a living from web analytics software

money In another interesting post on his blog, Avinash questions the conventional wisdom about "Enterprise Class" software, specifically web analytics software. The post made me smile because of Avinash's hyper-cautious preamble about how he doesn't set out to be contentious, but sometimes he just has to speak his mind, yada yada yada. I had a mental picture of him donning helmet, mitts, shin guards and chest protectors (not to mention the other thing) before continuing with the rest of his post.

But in the end Avinash has not had to endure a hailstorm of foul tips (see how I worked that in? You'd almost believe I had the faintest idea what I was talking about); his comments box is full of people agreeing with him that "Enterprise class" is a bogus term usually used by software companies to provide an excuse for charging a lot for software.

But Avinash's post did make me think about the bind that companies who want to charge a lot of money (or even any money at all) for web analytics are in - it's actually pretty difficult to build a business on the back of this kind of software.

Violins, please

If you're trying to build a business on the back of paid web analytics software, you have a problem. And that problem is that, however busy the E-metrics summit may get, and how many thousands of people read my blog on a daily basis (that's about 0.1 thousand, since you ask), web analytics remains a niche activity. This limits the total amount of money a web analytics vendor can get out of a specific customer.

Many of the worlds biggest software companies (and Microsoft is probably the prime example) have got there because they have been able to cut deals with customers to put their software on hundreds or thousands of desktops across the business. The numbers quickly multiply: a $100 desktop app (or OS)sold for 10,000 seats nets you a million dollars - and that's without implementation, server-side stuff, upgrades etc.

But this is never going to happen with web analytics, because the user base within the customer is just too small. I might go to (say) Coca Cola Corp and sell them a fancy web analytics implementation for maybe a million bucks, but once I'm done, what can I sell them next? Well, the first thing I do is make sure that I can charge them a million dollars next year as well - this, more than anything else, has driven the web analytics industry to an ASP model. And hopefully my company's product range has something else in it that I can sell into the customer this year. And, of course, I can sell service.


Service is great because it's the gift that keeps on giving (to the vendor, that is). Compared to going and finding a new customer, or even persuading an existing customer to buy something new, extending a service contract is easy peasy. But the problem with service is that it's low margin. Before you know it, the software company that you thought you ran has turned into a professional services company; your developers end up spending time out on the road with customers because it's hard to find and train consultants quickly enough, and it's even harder to schedule their time efficiently.

Before long, this reliance on service for revenues can have a detrimental effect on the product itself, causing it to become harder to use and thus more reliant on service. What it certainly does is provide a poor incentive for a software vendor to provide a solution that can get up and running quickly with the minimum help. It also provides a poor incentive to vendors to walk away from deals which will "run and run". When was the last time you heard a rep for a software vendor say, "you know, if we implement our software in your environment, it'll take ages to get value from it, and you'll end up spending four times as much on the implementation as you budgeted, so we we're not going to bid on this deal"?

So "Enterprise" web analytics companies face a number of options in order to keep growing - find more customers (difficult as the top end of market starts to really solidify), build or acquire new things to sell to their existing customers, find new ways of extracting more money from existing customers (sometimes referred to by the technical term, "price gouging"), or find other ways of earning money (for example, by using ad revenues to subsidize product development).

This is why you get such heartburn (as a paid web analytics customer) when your traffic goes up and so does your web analytics bill. It also explains why there's a slide in every Omniture presentation which is headed "How we will take over every aspect of your business". And, of course, it's why, if you buy web analytics software, you can expect to have a very close relationship with your account manager.

Disclaimer/please don't hit me

You might think that my position here at Microsoft makes me biased against paid vendors, and that it's easy for me to take a pop when we don't have to charge for Gatineau. I can't change your mind about that, but let me just say that I have a great deal of respect for all web analytics tools out there, and think that a paid model is an essential part of the industry. My remarks in this post are borne from six long years of trying to make web analytics pay in an Enterprise vendor. It's hard work.

del.icio.usdel.icio.us diggDigg RedditReddit StumbleUponStumbleUpon


About me



Enter your email address:

Delivered by FeedBurner