The rise of navigational search

There's a very interesting post by Robin Goad over on the Hitwise blog about the change in distribution of search terms on search engines. As more and more people are searching online, two things are happening to the range of search terms, one intuitively obvious, the other somewhat counter-intuitive.

The obvious change is that the total number of different search terms used is going up. If you think of the distribution of search terms as a head/tail curve, this means that the tail is getting longer. The non-obvious change is that the number of terms that make up the top 5, 10 or 20% of all searches (i.e. the most popular terms, or the "head") is going down.

Robin's charts are a little tough to parse, so I've taken a crack at simplifying them a little. Firstly, the chart below shows the change in breadth of search terms in the top 5%, 10% and 20% of search traffic. I've normalized the vertical axis (2005 = 100) to highlight the proportional change. You can see that in the top 5% group, the overall number of search terms fallen by over 80%.

This implies that a few very popular search terms are really starting to dominate the traffic. Robin goes a stage further and separates out "navigational" search terms, to produce the following chart. Here the drop-off in diversity is even more marked (note that the buckets in the chart below are different to those in the one above), with an average drop-off of around 80% even up to the 10% point.

What do we mean by navigational search? Searches for sites by their own name; for example, people trying to find the British Airways site by searching on "British Airways".

What this data tells us is that these brand or navigational search terms are starting to crowd the top of the leaderboard for searches, with people using them as proxies for remembering the URL of the site itself.

The reason this is interesting to online marketers is that sites are increasingly having to pay attention to these search terms - and some sites are choosing to buy their own company name as paid search to ensure that visitors click through to their site when they search on their company name. Look again at the search results (linked above) for "British Airways". The first sponsored results is... an ad for British Airways, despite the fact that BA's site is first in the Organic results, just a couple of lines down.

Apart from the fact that having to do this is likely costing BA a fair amount of money (which they could instead be spending on me when I fly to London at the weekend), it's likely to skew BA's picture of how their online marketing mix is really working for them. A lot of users who have been researching online (especially for something like flights) will use a navigational search term to return to the site where they've decided to purchase. Because most analytics tools use a "last-click" attribution model for conversions, BA's reporting on marketing effectiveness is likely to overstate the relative importance of those keywords, when it may really have been other keywords (or other kinds of marketing altogether, such as e-mail) which drove the visitor to the site in the first place.

So what are brand owners to do in this situation? They don't want to drop their navigational search keyword campaigns because they'll lose clicks and business, but on the other hand, buying navigational terms seems like a bit of a tax for these kind of sites, and distorts the numbers. Part of the answer lies in the rules the search engines impose about bidding on other companies' brand names (though this has caused all sorts of misery with Live Search and adCenter), but the true answer lies in a smarter attribution model for the sites involved.

In addition, sites should group branded or navigational search into a separate bucket, and take conversions that are attributed to those terms with a pinch of salt. I would even recommend that these conversions be not included when calculating the overall ROI of paid search, and instead be thought of as part of the cost of more brand-focused marketing activities such as TV. What do you think?