Lies, Damned Lies...

The Electrification of Marketing

At the tail end of the nineteenth century, electricity was starting to have a profound effect on the world. As dramatized in the excellent novel The Last Days of Night, and shortly in the forthcoming film The Current War, Thomas Edison battled with George Westinghouse (the latter aided by Croatian genius/madman Nikola Tesla) for control over the burgeoning market for electricity generation and supply. The popular symbol of the electrical revolution is of course Edison’s famous light bulb, but perhaps almost more important was the humble electric motor.

The electric motor was so important because it revolutionized manufacturing, enabling factories to create assembly lines and realize huge efficiency dividends. The Ball Brothers Glass Manufacturing Company, for example, replaced 36 workers with a single electric crane for moving heavy loads across the factory where they made their famous Mason jars.

But for all the benefits of electric motors, many factories were slow to embrace the new technology. As this article from the BBC World Service’s “50 Things that Made the Modern Economy” podcast explains, by 1900, almost twenty years after Thomas Edison started selling electricity from his generation plants in Manhattan and London, only 5% of factories had switched from steam to electric power. Powering a factory with a steam engine was costly, complicated, and dangerous. So why the reluctance to move to electricity?

The reason lies in the way those factories were organized to take advantage of steam power generation. A typical nineteenth century factory, for example making textiles, looked like the image above. Mechanical power was generated by a single large steam engine which ran more or less continuously, and was transferred to individual machines (such as looms or lathes) via a series of drive shafts, gears and drive belts. Because the power was being transferred mechanically, the machines were packed closely together. This, combined with the constant spinning of the drive shafts, made these factories very dangerous to work in; in 1900, over half a million people in the US (almost 1% of the population) were maimed in factory accidents.

Simply replacing the central steam engine with an electric motor did not deliver significant benefits – the drive shafts and belts to the machines still broke down, factories were still crowded, inefficient and dangerous, and the central motor (now powered by comparatively expensive electricity) still had to be kept running constantly.

To truly capitalize on electrification, factories had to reinvent themselves, replacing all their individual machines with versions that were powered by their own electric motors, with power transferred to them via unobtrusive wires rather than spinning drive shafts. In turn this meant that machines did not need to be so tightly packed together; factories could be reorganized to be more spacious and facilitate the flow of items, paving the way for the production line and improving factory conditions and safety. Ultimately, it was the qualitative transformation in the way things were made which was electrification’s biggest benefit.

Reorganizing the marketing factory

The story of electrification and how it impacted manufacturing in the first decades of the twentieth century provides an interesting parallel to the impact of data and AI on the marketing industry in the first decades of the twenty-first.

Today, many marketing organizations have adopted data in a similar way to how factories first adopted electricity: by applying it to existing business processes and ways of working. In direct marketing, the core processes of list-generation and campaign delivery have not changed fundamentally in fifty years – marketers build target audience lists, map messages to this list, deliver those messages, and then measure the response. The sophistication and complexity of all these steps has changed dramatically, but the process itself is still the same.

However, as electricity led to the development of new kinds of manufacturing machines, so data is leading the the development of new kinds of marketing machines, powered by AI. These new systems, which I have written about before, promise to transform the way that digital marketing is done. But just as before, getting there won’t be easy, and will require marketing leaders to embrace disruptive change.

The current ‘factory layout’ for many marketing organizations is based around individual teams that have responsibility for different channels, such as web, search, email, mobile and so on. These teams coordinate on key marketing calendar activities, such as holiday campaigns or new product launches, but manage their own book of work as a sequence of discrete activities. At Microsoft we’ve made progress in the last few years on bringing many of these teams together, and supporting them with a common set of customer data and common marketing automation tooling. But individual campaigns are still largely hand-crafted.

AI-driven marketing systems use a wide range of attributes at the customer level, combined with a continuous testing/learning approach, to discover which of a range of creative and messaging should be executed next, for which customers, in which channels. They break down the traditional campaign-centric model of customer communications and replace it with a customer-centric, ‘always on’ program of continuous nurture. For these systems to work well, they need a detailed picture of the customer, including their exposure and response to previous communications, and they need a wide range of actions that they can take, including the ability to choose which channel to communicate in for a given message and audience.

A fairly traditional marketing organization that is looking to evaluate the potential of AI-driven marketing will, prudently, lean towards trying the technology in a relatively limited pilot environment, likely choosing just one campaign or program in a single channel for their test. These choices make sense – few companies can easily try out new technology across multiple channels, for both technical reasons (i.e. wiring the thing up) and organizational reasons (getting multiple teams to work together).

But this approach is a bit like a 1900’s factory owner deciding to replace just a single machine in the factory with an electric version. Dedicated (and expensive) wiring would have to be laid to power the machine; it would still be crammed in with all the others, so its size and design would be limited; and it would likely need a dedicated operator. In this environment, it would be unlikely that the single machine would be so transformatively efficient that the factory owner would rush out to buy twenty more.

And so it is with AI-driven marketing. A test within a single channel, on a single campaign, will likely generate modest results, because the machine’s view of the customer will be limited to their experience with that brand in that channel; its message choices will also be limited, since it can only communicate within the single channel. These problems are exacerbated by the expense of laying dedicated data ‘lines’ to the new system, and of building many creative variants, to give the system enough message choice within a single channel.

What’s needed is for AI-based optimization to be applied as an enabling capability in all marketing campaigns, across multiple channels and products. This requires significant investment in data and channel integration; but even more importantly it requires marketers, and marketing organizations, to operate differently. Digital advertising, CRM and e-commerce teams, and their budgets, need to be brought together; instead of marketers creating many discrete campaigns, marketers need to create more evergreen programs that can be continuously optimized over time. The marketing factory needs to be organized around the customer, not the product or channel.

This kind of model represents very disruptive change for today’s marketing organizations, as it did for yesterday’s factory owners. In the end, much of the rise of electrified factories a hundred years ago was due to the efforts of newcomers to the field such as Henry Ford, who jumped straight to an electrified production line in the production of his Model T. Today’s marketing chiefs would do well to heed this lesson from history, as disruptors like Amazon, Tesla and Stitch Fix use process innovation to create streamlined, customer-centric marketing functions that are poised to exploit the transformative technology of AI.

Is Digital Marketing having its ‘Deep Blue’ moment?

Garry Kasparov will forever be remembered as perhaps the greatest chess player of all time, dominating the game for almost twenty years until his retirement in 2005. But ironically he may be best remembered for the match he failed to win twenty years ago in 1997 against IBM’s Deep Blue chess computer. That watershed moment – marking the point at which computers effectively surpassed humans in chess-playing ability – prompted much speculation and hand-wringing about the coming obsolescence of the human brain, now that a mere computer had been able to beat the best chess grandmaster in the world.

Since then, computers and chess software have only grown more powerful, to the point that a $50 commercial chess program (or even a mobile app) can beat most grandmasters easily. Faced with this, you might expect Kasparov and other top-flight players to have grown disillusioned with the game, or defensive about the encroachment of computers on their intellectual territory; but in fact the reverse is true.

Today’s chess grandmasters make extensive use of computers to practice, try out new strategies, and prepare for tournaments, in the process becoming a little more like the machines that outpaced them in 1997. Kasparov himself was instrumental in pioneering a  new type of chess game, Advanced Chess, in which humans are allowed to consult with chess software as they play. In his new book, “Deep Thinking: Where Machine Intelligence Ends and Human Intelligence Begins”, Kasparov writes about an Advanced Chess match he played in 1998 against Veselin Topalov:

“Having a computer partner also meant never having to worry about making a tactical blunder. The computer could project the consequences of each move we considered, pointing out possible outcomes and countermoves we might otherwise have missed. With that taken care of for us, we could concentrate on strategic planning instead of spending so much time on calculations. Human creativity was even more paramount under these conditions.”

What Kasparov and his successors in the competitive chess-playing world have discovered was that, when it comes to chess, the strongest player is not man or machine, but man and machine. In fact, a new kind of chess tournament has sprung up, Freestyle Chess, in which teams of humans and computers compete against one another, each bringing their respective strengths to the game: creativity, strategy and intuition from the humans, and tactical outcome prediction from the computers.

And your point is?

You may be asking what relevance this has to digital marketing. In fact, there are strong similarities between chess and marketing (particularly digital marketing):  they are both highly quantifiable pursuits with clear outcomes which have historically relied solely on human intuition and creativity for success.

As in chess, digital marketing relies upon a continuous reassessment of the ‘board’ (customer behaviors and history) in order to decide upon the next ‘move’ (a particular campaign communication aimed at a particular group of customers). Once the move has been made, the board needs to be reassessed before taking the next move.

Today’s digital marketer is much like the chess grandmaster of the early 1990s – they rely on their intuitive understanding of their audience’s makeup and preferences to decide what offers and messages they want to deliver, to which users, and in which channels. Of course, digital marketers understand that measuring campaign outcomes and audience response (using techniques like control groups and attribution analysis) is very important, but most still operate in a world where the humans make the decisions, and the computers merely provide the numbers to support the decision-making.

Luddites 2.0

When Kasparov was asked in 1990 if a computer could beat a grandmaster before the year 2000, he quipped:

“No way - and if any grandmaster has difficulties playing computers, I would be happy to provide my advice.”

Today’s digital marketers can be forgiven for exhibiting some of the same skepticism. Ask them how they came up with a new idea for an ad, or how they know that a particular product will be just right for a particular audience, and they may not be able to answer – they will just know that their intuition is sound. As a result it can seem incredible that a computer can pick the right audience for a campaign, and match the appropriate offer and creative to that audience. 

But the computers are coming. As I mentioned in my earlier post on bandit experimentation, companies like Amplero, Kahuna and Cerebri AI are pitching intelligent systems that claim to take a lot of this decision-making about creative choice, audience, channel and other campaign variables out of the hands of humans. But where does that leave the digital marketer?

We welcome our robot colleagues

The clue lies in the insights that Kasparov ultimately drew from his defeat. He realized that the strengths he brought were different and complementary to the strengths of the computer. The same holds true for digital marketing. Coming up with product value propositions, campaign messaging and creative are activities which computers are nowhere close to being good at, especially in the context of broader intangible brand attributes. On the other hand, audience selection and targeting, as well as creative optimization, are highly suited to automation, to the extent that computers can be expected to perform significantly better than their human counterparts, much as chess software outperforms human players.

Clearly humans and machines need to work together to create and execute the best performing campaigns, but exactly how this model will work is still being figured out.

Today, most digital marketers build campaign audiences by hand, identifying specific audience attributes (such as demographics or behavioral history) and applying filters to to those attributes to build segments. The more sophisticated the marketer attempts to be in selecting audience attributes for campaign segments, the more cost they incur in the setup of those campaigns, making the ROI equation harder to balance.

The emerging alternative approach is to provide an ML/AI system with a set of audience (and campaign) attributes, and let it figure out which combinations of audience and offer/creative deliver the best results by experimenting with different combinations of these attributes in outbound communications. But this raises some important questions:

• How to choose the attributes in the first place
• How to understand which attributes make a difference
• How to fit ML/AI-driven campaigns into a broader communications cadence & strategy
• How to use learnings from ML/AI-driven campaigns to develop new value propositions and creative executions

In other words, ML/AI-driven marketing systems cannot simply be ‘black boxes’ into which campaign objectives and creative are dumped, and then left to deliver clicks or conversions on the resulting campaign delivery. They need to inform and involve marketers as they do their work, so that the marketers can make their uniquely human contribution to the process of designing effective campaigns. The black box needs some knobs and dials, in other words.

The world of chess offers a further useful parallel here. Chess grandmasters make extensive use of specialized chess software like Fritz 15 or Shredder, which not only provide a comprehensive database of chess moves, but also training and analysis capabilities to help human players improve their chess and plan their games. These programs don’t simply play chess – they explain how they are making their recommendations, to enable their human counterparts to make their own decisions more effectively.

These are the kinds of systems that digital marketers need to transform their marketing with AI. In turn, marketers need to adjust the way they plan and define campaigns in the same way that chess grandmasters have dramatically changed the way they study, plan and play games of chess in the last twenty years, working alongside the computers before, during and after campaigns are run.

In 1997, it was far from clear how chess, and the people who played it , would react to the arrival of computers. Digital Marketing stands on a similar threshold today. Twenty years from now it will seem obvious how marketers’ roles would evolve, and how technology would adapt to support them. We’re in the fortunate position of getting to figure this out as it all unfolds, much as Kasparov did.

6 steps to building your Marketing Data Strategy

Your company has a Marketing Strategy, right? It’s that set of 102 slides presented by the CMO at the offsite last quarter, immediately after lunch on the second day, the session you may have nodded off in (it’s ok, nobody noticed. Probably). It was the one that talked about customer personas and brand positioning and social buzz, and had that video towards the end that made everybody laugh (and made you wake up with a start).

Your company may also have a Data Strategy. At the offsite, it was relegated to the end of the third day, after the diversity session and that presentation about patent law. Unfortunately several people had to leave early to catch their flights, so quite a few people missed it. The guy talked about using Big Data to drive product innovation through continuous improvement, and he may (at the very end, when your bladder was distracting you) have mentioned using data for marketing. But that was something of an afterthought, and was delivered with almost a sneer of disdain, as if using your company’s precious data for the slightly grubby purpose of marketing somehow cheapened it.

Which is a shame, because Marketing is one of the most noble and enlightened ways to use data, delivering a direct kick to the company’s bottom line that is hard to achieve by other means. So when it comes to data, your marketing shouldn’t just grab whatever table scraps it can and be grateful; it should actually drive the data that you produce in the first place. This is why you don’t just need a Marketing Strategy, or a Data Strategy: You need a Marketing Data Strategy.

A Marketing Data What?

What even is a Marketing Data Strategy, anyway? Is it even a thing? It certainly doesn’t get many hits on Bing, and those hits it does get tend to be about building a data-driven Marketing Strategy (i.e. a marketing strategy that focuses on data-driven activities). But that’s not what a Marketing Data Strategy is, or at least, that’s not my definition, which is:

A Marketing Data Strategy is a strategy for acquiring, managing, enriching and using data for marketing.

The four boldface words are the key here. If you want to make the best use of data for your marketing, you need to be thinking about how you can get hold of the data you need, how you can make it as useful as possible, and how you can use your marketing efforts themselves to generate even more useful data – creating a positive feedback loop and even contributing to the pool of Big Data that your Big Data guy is so excited about turning into an asset for the company.

Building your Marketing Data Strategy

So know that you know why it’s important to have a Marketing Data Strategy, how do you put one together? Everyone loves a list, so here are six steps you can take to build and then start executing on your Marketing Data Strategy.

Step 1: Be clear on your marketing goals and approach

This seems obvious, but it’s a frequently missed step. Having a clear understanding of what you’re trying to achieve with your digital marketing will help you to determine what data you need, and what you need to do with/to it to make it work for you. Ideally, you already have a marketing strategy that captures a lot of this, though the connection between the lofty goals of a marketing strategy (sorry, Marketing MBA people) and the practical data needs to execute the strategy are not always clear.

Here are a few questions you should be asking:

Get new customers, or nurture existing ones? If your primary goal is to attract new customers, you’ll need to think differently about data (for example relying on third-party sources) than if you are looking to deepen your relationship with your existing customers (about whom you presumably have some data already).

What are your goals & success criteria? If you are aiming to drive sales, are you more interested in revenue, or margin? If you’re looking to drive engagement or loyalty, are you interested in active users/customers, or engagement depth (such as frequency of usage)?

Which communications strategies & channels? The environments in which you want to engage your audience make a big difference to your data needs – for example, you may have more data at your disposal to target people using your website compared to social or mobile channels.

Who’s your target audience? What attributes identify the people you’d most like to reach with your marketing? Are they primarily demographic (e.g. gender, age, locale) or behavioral (e.g. frequent users, new users)?

What is your conversion funnel? Can you convert customers entirely online, or do you need to hand over to humans (e.g. in store) at some point? If the latter, you’ll need a way to integrate offline transaction data with your online data.

These questions will not only help you identify the data you’ll need, but also some of the data that you can expect to generate with your marketing.

Step 2: Identify the most important data for your marketing efforts

Once you’re clear on your goals and success criteria, you need to consider what data is going to be needed to help you achieve them, and to measure your success.

The best way to break this down is to consider which events (or activities) you need to capture and then which attributes (or dimensions) you need on those events. But how to pick the events and attributes you need?

Let’s start with the events. If your marketing goals include driving revenue, you will need revenue (sales) events in your data, such as actual purchase amounts. If you are looking to drive adoption, then you might need product activation events. If engagement is your goal, then you will need engagement events – this might be usage of your product, or engagement with your company website or via social channels.

Next up are the attributes. Which data points about your customers do you think would be most useful for targeted marketing? For example, does your product particularly appeal to men, or women, or people within a certain geography or demographic group?

For example, say you’re an online gambling business. You will have identified that geo/location information is very important (because online gambling is banned in some countries, such as the US). Therefore, good quality location information will be an important attribute of your data sources.

At this step in the process, try not to trip yourself up by second-guessing how easy or difficult it will be to capture a particular event or attribute. That’s what the next step (the data audit) is for.

Step 3: Audit your data sources

Now to the exciting part – a data audit! I’m sure the very term sends shivers of anticipation down your spine. But if you skip this step, you’ll be flying blind, or worse, making costly investments in acquiring data that you already have.

The principle of the data audit is relatively simple – for every dataset you have which describes your audience/customers and their interaction with you, write down whether (and at what kind of quality) they contain the data you need, as identified in the previous step:

• Events (e.g. purchases, engagement)
• Attributes (aka dimensions, e.g. geography, demographics)
• IDs (e.g. cookies, email addresses, customer IDs)

The key to keeping this process from consuming a ton of time and energy is to make sure you’re focusing on the events, attributes and IDs which are going to be useful for your marketing efforts. Documenting datasets in a structured way is notoriously challenging (some of the datasets we have here at Microsoft have hundreds or even thousands of attributes), so keep it simple, especially the first time around – you can always go back and add to your audit knowledge base later on.

The one type of data you probably do want to be fairly inclusive with is ID data. Unless you already have a good idea which ID (or IDs) you are going to use to stitch together your data, you should capture details of any ID data in your datasets. This will be important for the next step.

To get you started on this process, I’ve created a very simple data audit template which you can download here. You’re welcome.

Step 4: Decide on a common ID (or IDs)

This is a crucial step. In order for you to build a rich profile of your users/customers that will enable you to target them effectively with marketing, you need to be able to stitch the various sources of data about them together, and for this you need a common ID.

Unless you’re spectacularly lucky, you won’t be issuing (or logging) a single ID consistently across all touchpoints with your users, especially if you have things like retail stores, where IDing your customers reliably is pretty difficult (well, for the time being, at least). So you’ll need to pick an ID and use this as the basis for a strategy to stitch together data.

When deciding which ID or IDs to use, take into consideration the following attributes:

• The persistence of the ID. You might have a cookie that you set when people come visit your website, but cookie churn ensures that that ID (if it isn’t linked to a login) will change fairly regularly for many of your users, and once it’s gone, it won’t come back.
• The coverage of the ID. You might have a great ID that you capture when people make a purchase, or sign up for online support, but if it only covers a small fraction of your users, it will be of limited use as a foundation for targeted marketing unless you can extend its reach.
• Where the ID shows up. If your ID is present in the channels that you want to use for marketing (such as your own website), you’re in good shape. More likely, you’ll have an ID which has good representation in some channels, but you want to find those users in another channel, where the ID is not present.
• Privacy implications. User email address can be a good ID, but if you start transmitting large numbers of email addresses around your organization, you could end up in hot water from a privacy perspective. Likewise other sensitive data like Social Security Numbers or credit card numbers – do not use these as IDs.
• Uniqueness to your organization. If you issue your own ID (e.g. a customer number) that can have benefits in terms of separating your users from lists or extended audiences coming from other providers; though on the other hand, if you use a common ID (like a Facebook login), that can make joining data externally easier later.

Whichever ID you pick, you will need to figure out how you can extend its reach into the datasets where you don’t currently see it. There are a couple of broad strategies for achieving this:

• Look for technical strategies to extend the ID’s reach, such as cookie-matching with a third-party provider like a DMP. This can work well if you’re using multiple digital touchpoints like web and mobile (though mobile is still a challenge across multiple platforms).
• Look for strategies to increase the number of signed-in or persistently identified users across your touchpoints. This requires you to have a good reason to get people to sign up (or sign in with a third-party service like Facebook) in the first place, which is more of a business challenge than a technical one.

As you work through this, make sure you focus on the touchpoints/channels where you most want to be able to deliver targeted messaging – for example, you might decide that you really want to be able to send targeted emails and complement this with messaging on your website. In that case, finding a way to join ID data between those two specific environments should be your first priority.

Step 5: Find out what gaps you really need to fill

Your data audit and decisions around IDs will hopefully have given you some fairly good indications of where you’re weak in your data. For example, you may know that you want to target your marketing according to geography, but have very little geographic data for your users. But before you run off to put a bunch of effort into getting hold of this data, you should try to verify whether a particular event or attribute will actually help you deliver more effective marketing.

The best way to do this is to run some test marketing with a subset of your audience who has a particular attribute or behavior, and compare the results with similar messaging to a group who which does not have this attribute (but are as similar in other regards as you can make them). I could write another whole post on this topic of A/B testing, because there is a myriad of ways that you can mess up a test like this and invalidate your results, or I could just recommend you read the work of my illustrious Microsoft colleague, Ronny Kohavi.

If you are able to run a reasonably unbiased bit of test marketing, you will discover whether the datapoint(s) you were interested in actually make a difference to marketing outcomes, and are therefore worth pursuing more of. You can end up in a bit of a chicken-and-egg situation in this regard, because of course you need data in the first place to test its impact, and even if you do have some data, you need to test over a sufficiently large population to be able to draw reliable conclusions. To address this, you could try working with a third-party data provider over a limited portion of your user base, or over a population the provider provides.

Step 6: Fix what you can, patch what you can’t, keep feeding the beast

Once you’ve figured out which data you actually need and the gaps you need to fill, the last part of your Marketing Data Strategy is about tactics to actually get this data. Of course the tactics then represent an ongoing (and never-ending) process to get better and better data about your audience. Here are four approaches you can use to get the data you need:

Measure it. Adding instrumentation to your website, your product, your mobile apps, or other digital touchpoints is (in principal) a straightforward way of getting behavioral events and attributes about your users. In practice, of course, a host of challenges exist, such as actually getting the instrumentation done, getting the signals back to your datacenter, and striking a balance between well-intentioned monitoring of your users and appearing to snoop on them (we know a little bit about the challenges of striking this balance).

Gather it. If you are after explicit user attributes such as age or gender, the best way to get this data is to ask your users for it. But of course, people aren’t just going to give you this information for no reason, and an over-nosy registration or checkout form is a sure-fire way to increase drop-out from your site, which can cost you money (just ask Bryan Eisenberg). So you will need to find clever ways of gathering this data which are linked to concrete benefits for your audience.

Model it. A third way to fill in data gaps is to use data modeling to extrapolate attributes that you have on some of your audience to another part of your audience. You can use predictive or affinity modeling to model an existing attribute (e.g. gender) by using the behavioral attributes of existing users whose gender you know to predict the gender of users you don’t know; or you can use similar techniques to model more abstract attributes, such as affinity for a particular product (based on signals you already have for some of your users who have recently purchased that product). In both cases you need some data to base your models on and a large enough group to make your predictions reasonably accurate. I’ll explore these modeling techniques in another post.

Buy it. If you have money to spend, you can often (not always) buy the data you need. The simplest (and crudest) version of this is old-fashioned list-buying – you buy a standalone list of emails (possibly with some other attributes) and get spamming. The advantage of this method is that you don’t need any data of your own to go down this path; the disadvantages are that it’s a horrible way to do marketing, will deliver very poor response rates, and could even damage your brand if you’re seen as spamming people. The (much) better approach is to look for data brokers that can provide data that you can join to your existing user/customer data (e.g. they have a record for user [email protected] and so do you, so you can join the data together using the email address as a key).

Once you’ve determined which data makes the most difference for your marketing, and have hit upon a strategy (or strategies) to get more of this data, you need to keep feeding the beast. You won’t get all the data you need – whether you’re measuring it, asking for it, or modeling it – right away, so you’ll need to keep going, adjusting your approach as you go and learn about the quality of the data you’re collecting. Hopefully you can reduce your dependency on bought data as you go.

Finally, don’t forget – all this marketing you’re doing (or plan to do) is itself a very valuable source of data about your users. You should make sure you have a means to capture data about the marketing you’re exposing your users to, and how they’re responding to it, because this data is useful not just for refining your marketing as you go along, but can actually be useful other areas of your business such as product development or support. Perhaps you’ll even get your company’s Big Data people to have a bit more begrudging respect for marketing…

The rise of the Chief Data Officer

As the final season of Mad Men came to a close this weekend, one of my favorite memories from Season 7 is the appearance of the IBM 360 mainframe in the Sterling Cooper & Partners offices, much to the chagrin of the creative team (whose lounge was removed to make space for the beast), especially poor old Ginsberg, who became convinced the “monolith” was turning him gay (and took radical steps to address the issue).

My affection for the 360 is partly driven by the fact that I started my career at IBM, closer in time to Man Men Series 7 (set in 1969) than the present day (and now I feel tremendously old having just written that sentence). The other reason I feel an affinity for the Big Blue Box is because my day job consists of thinking of ways to use data to make marketing more effective, and of course that is what the computer at SC&P was for. It was brought in at the urging of the nerdish (and universally unloved) Harry Crane, to enable him to crunch the audience numbers coming from Nielsen’s TV audience measurement service to make TV media buying decisions. This was a major milestone in the evolution of data-driven marketing, because it linked advertising spend to actual advertising delivery, something that we now take for granted.

The whole point of Mad Men introducing the IBM computer into the SC&P offices was to make a point about the changing nature of advertising in the early 1970s – in particular that Don Draper and his “three martini lunch” tribe’s days were numbered. Since then, the rise of the Harry Cranes, and the use of data in marketing and advertising, has been relentless. Today, many agencies have a Chief Data Officer, an individual charged with the task of helping the agency and its clients to get the best out of data.

But what does, or should, a Chief Data Officer (or CDO) do? At an advertising & marketing agency, it involves the following areas:

Enabling clients to maximize the value they get from data. Many agency clients have significant data assets locked up inside their organization, such as sales history, product telemetry, or web data, and need help to join this data together and link it to their marketing efforts, in order to deliver more targeted messaging and drive loyalty and ROI. Additionally, the CDO should advise clients on how they can use their existing data to deliver direct value, for example by licensing it.

Advising clients on how to gather more data, safely. A good CDO offers advice to clients on strategies for collecting more useful data (e.g. through additional telemetry), or working with third-party data and data service providers, while respecting the client’s customers’ privacy needs.

Managing in-house data assets & services. Some agencies maintain their own in-house data assets and services, from proprietary datasets to analytics services. The CDO needs to manage and evolve these services to ensure they meet the needs of clients. In particular, the CDO should nurture leading-edge marketing science techniques, such as predictive modeling, to help clients become even more data-driven in their approach.

Managing data partnerships. Since data is such an important part of a modern agency’s value proposition, most agencies maintain ongoing relationships with key third-party data providers, such as BlueKai or Lotame.The CDO needs to manage these relationships so that they complement the in-house capabilities of the agency, and so the agency (and its clients) don’t end up letting valuable data “walk out of the door”.

Driving standards. As agencies increasingly look to data as a differentiating ingredient across multiple channels, using data and measurement consistently becomes ever more important. The CDO needs to drive consistent standards for campaign measurement and attribution across the agency so that as a client works with different teams, their measurement framework stays the same.

Engaging with the industry & championing privacy. Using data for marketing & advertising is not without controversy, so the DCO needs to be a champion for data privacy and actively engaged with the industry on this and other key topics.

As you can see, that’s plenty for the ambitious CDO to do, and in particular plenty that is not covered by other traditional C-level roles in an ad agency. I think we’ll be seeing plenty more CDOs appointed in the months and years to come.

Should Wikipedia accept advertising?

It’s that time of year again. The nights are drawing in, snow is starting to fall in the mountains, our minds turn to thoughts of turkey and Christmas pudding, and familiar faces appear: Santa, Len and Bruno, and of course, Jimmy Wales.

If you are a user of Wikipedia (which, if you’re a user of the Internet, you almost certainly are), you’ll likely be familiar with Jimmy Wales, the founder of Wikipedia and head of the Wikimedia Foundation, the non-profit which runs the site. Each year Jimmy personally fronts a campaign to raise funds to cover the cost of running Wikipedia, which this year will amount to around $29m.

The most visible part of this campaign is the giant banner featuring Jimmy Wales’s face which appears at the top of every Wikipedia article at this time of year. This year the banner has caused some hilarity as the position of the picture of Jimmy just above the article title has provided endless comic potential (as above), but every year it becomes increasingly wearisome to have Jimmy’s mug staring out at you for around three months. Would it not be easier for all concerned if Wikipedia just carried some advertising?

Jimmy has gone on record as saying that he doesn’t believe that Wikipedia should be funded by advertising, and I understand his position. To parse/interpret his concerns, I believe he’s worried about the following:

• Accepting advertising would compromise Wikipedia’s editorial independence from commercial interests
• Ads would interfere with the user experience of Wikipedia and be intrusive
• Wikipedia contributors would not want to contribute for free to Wikipedia if they knew it was accepting advertising

I’m biased, of course, since I work for Microsoft Advertising, but I believe that each of these concerns is manageable. Let’s take them one by one:

Concern 1: Ads would compromise Wikipedia’s independence

There are plenty of historical examples where a publication has been put in a difficult position when deciding what to publish because of relationships with large advertisers. Wikipedia certainly doesn’t want, for example, Nike complaining about the content of its Wikipedia entry. And the idea of Wikipedia starting to employ sales reps to hawk its inventory is a decidedly unedifying one.

But Wikipedia does not have to engage in direct sales, or even non-blind selling, to reach its financial goals with advertising. The site could make its inventory available on a blind ad network (or ideally multiple networks) so that it would be impossible for an advertiser to specifically buy ad space on Wikipedia. If an advertiser didn’t like their ads appearing on Wikipedia, most networks offer a site-specific opt out, but the overall impact of this to Wikipedia would be minimal – Wikipedia carries such a vast range of content that it has the most highly diversified content portfolio in the world – no single advertiser could exert any real leverage over it.

Concern 2: Ads would make Wikipedia suck

As has been noted elsewhere, there are plenty of horrible ads at large in the Internet – intrusive pop-ups, or horrible creative. It would certainly be a valid concern that Wikipedia would suddenly become loaded with distracting commercial messages. But according to the back-of-an-envelope calculations I’ve done, there is no need for Wikipedia to saturate itself with ads in order to pay the bills.

According to the excellent stats.wikimedia.org site, Wikipedia served almost exactly 15bn page views world-wide in October 2011 (around half of which were in English). Assuming no growth in that figure over 12 months, that’s around 180bn PVs per year. So to meet its funding requirements, Wikipedia would need to generate a $0.16 eCPM on those page views (assuming just one ad unit per page). That’s a pretty modest rate, especially on a site with as much rich content as Wikipedia. It would give the site a number of options in terms of ad placement strategy, such as:

• Place a very low-impact, small text ad on every page
• Place a somewhat larger/more impactful ad on a percentage of pages on a rotation, and leave other pages ad free
• Place ads on certain types of pages, leaving others always ad free (such as pages about people or companies, or pages in a particular language/geo)
• Deploy a mix of units across different types of page, or in rotation

This also assumes that Wikimedia needs to raise all its funds every year from advertising, which it may not need to – though once the site accepted advertising, it would definitely become more difficult (though perhaps not impossible) to raise donations.

To preserve the user experience, I would definitely recommend just running text ads, which could be placed relatively unobtrusively. Sites running text-based contextual ads (such as those from Google AdSense or Microsoft adCenter) can usually expect to get at least around $0.30 eCPM, so there would be some headroom.

I would also recommend that Wikipedia not run targeted ads – or at least, only work with networks that do not sell user data to third parties. It could cause significant backlash if it became felt that Wikipedia was effectively selling data about its users’ browsing habits to advertisers for a fast buck.

Concern 3: Ads would make contributors flee

I can speak to this concern less authoritatively, since I am not that familiar with the world of Wikipedia contribution, but so long as Wikimedia made it clear that it was remaining a non-profit organization, and continued to operate in a thrifty fashion to cover its costs, the initial outrage of Wikipedia contributors could be managed. After all, plenty of other open-source projects that rely on unpaid contributors do provide the foundations for commercial activities, Linux being the best example.

In any case, in its deliberations about balancing the needs of its contributors with its need to pay the bills, Wikimedia will need to face some hard questions: Will it always be able to cover its costs through donations? Does the current level of investment in infrastructure represent an acceptable level of risk for a site that serves so many users? Is it acceptable to rely on unpaid contributors indefinitely? If Wikipedia ran out of cash or went down altogether, the righteous indignation of its contributors may not count for very much.

Apart from advertising and donations, the only other way that Wikipedia could pay the bills would be by creating paid-for services – for example, a research service. But would the unpaid Wikipedia contributors really be happier with this outcome than with advertising? It would effectively amount to selling the content that they’d authored for free. At least with advertising, it’s the user that is the product, not the content. So long as Wikipedia can maintain editorial independence and retain a good user experience, advertising feels like the better option to me.

Does Display help Search? Or does Search help Display?

One of the topics that we didn’t get quite enough time to cover in detail in my face-off with Avinash Kaushik at last week’s eMetrics Summit (of which more in another post) was the thorny issue of conversion attribution. When I asked Avinash about it, he made the sensible point that trying to correctly “attribute” a conversion to a mix of the interactions that preceded it ends up being a very subjective process, and that adopting a more experimental approach – tweaking aspects of a campaign and seeing which tweaks result in higher conversion rates – is more sound.

I asked the question in part because conversion attribution is conspicuously absent from Google Analytics – a fact which raises an interesting question about whether it’s in Google’s interest to include a feature like this, since it may stand to lose more than it gains by doing so (since the effective ROI of search will almost certainly go down when other channels are mixed into an attribution model).

Our own Atlas Institute is quite vocal on this topic, and has published a number of white papers such as this one [PDF] about the consideration/conversion funnel, and this one [PDF], on which channels are winners and losers in the new world of Engagement Mapping (our term for multi-channel conversion attribution).

The Atlas Institute has also opined about how adding display to a search campaign can raise the effectiveness of that campaign by 22% compared to search alone – in other words, how display helps search to be better.

However, a recent study from iProspect throws some new light on this discussion. The study – a survey of 1,575 web consumers – attempted to discover how people respond to display advertising. And one of the most interesting findings from the study is that, whilst 31% of users claim to have clicked on a display ad in the last 6 months, almost as many – 27% – claimed that they responded to the ad by searching for that product or brand:

This raises the interesting idea that search can actually help display be better, by providing a response mechanism that differs from the traditional ad click behavior that we expect. Of course, this still doesn’t mean that search should get 100% of the credit for a conversion in this kind of scenario – in fact, it makes a stronger case for “view-through” attribution of display campaigns – something that ad networks (like, er, our own Microsoft Media Network) are keen to encourage people to do, to make performance-based campaigns look better.

All this really means that, of course, it’s not a case of display vs. search, but display and search (and a whole lot of other ways of reaching consumers). Whether you take the view that it’s your display campaign that helps your search to be more effective, or your search keywords that help your display campaign to drive more response, multi-channel online marketing – and the complexity that goes with measuring it – looks set for the big time. And by “big time”, I mean the army of small advertisers currently using systems like Google’s AdWords, or our own adCenter. So maybe we’ll see multi-channel conversion attribution in Google Analytics before long.

Self-serve outdoor advertising: Signposter

Outdoor advertising has not exactly been at the cutting edge of innovation in recent years, at least not in terms of broadening access to smaller advertisers. Whilst radio, TV and print advertising has been becoming easier to get into on a limited budget, getting your ad on a 96-sheet on the Cromwell Road has been rather harder.

So I was cheered to learn of a new service in the UK called Signposter which aims to make buying outdoor advertising as easy as buying paid search. Signposter's audience is small businesses who want to runs ad close to where they're based. The Signposter site has a nice little campaign planning tool (using some of our technology, pleasingly), allowing you to find outdoor units of various sizes (bus-sides, telephone boxes, and billboards), and then book space on them. I was quoted a price for a 48-sheet billboard in London of about £800 for a week ($1,600), but the price gets much more competitive for a two or three-week run (because, of course, a big chunk of the price is printing and sticking the poster up onto the billboard).

The other side of the site is an ad creation utility, allowing you to create an ad from a series of templates. I was rather disappointed by this tool, I must say - it's pretty clunky and slow, and seems to offer an unnecessarily limited range of customization options (for example, you can't upload your own artwork for a poster, except for a logo). Compared to amazing tools like Picnik, it seems primitive.

It may be that some of the restrictions are due to the agreements reached between Signposter and the billboard owners not to show offensive content; restricting the choices around images means that ads don't have to be manually approved before they're published (though people can still put in offensive copy).

The image at the top of this post represents my efforts to create an ad, with a cushion-obsessed retail client in mind. Not bad, eh? I could well imagine small retailers using a service like this to stand out from the crowd around Christmas-time.It will be interesting to see how the company fares as Christmas approaches - they're rolling the service out across the UK at the moment (they're in Newcastle, Birmingham and Portsmouth right now). Their success, apart from recruiting advertisers, will come from striking deals with the media owners themselves, and creating a kind of remnant inventory for outdoor advertising.

In the longer term, it will be interesting to see how digital billboards will affect Signposter's business model - there's a danger that someone like Google (or us) will come along and extend an existing ad buying system with digital billboards (or even non-digital ones). I wish them luck.

(Via Steve)

Is Google (or Microsoft) making you stupid?

There's a very interesting article in The Atlantic from Nicholas Carr in which he argues that the information delivery model of the web (and, in particular, the search engine model) is robbing us of our ability to read and digest information, and consequently affecting the very way we think. Whether this turns out to be a good or a bad thing for the human race remains to be seen (unfortunately, there's little chance of us being able to perform an A/B test on a segment of the population), but one paragraph of Nick's article (which, yes, I did read in its entirety, all the while looking at the clock on my computer thinking that I should be getting on with something else), stood out for me (my highlights):

The idea that our minds should operate as high-speed data-processing machines is not only built into the workings of the Internet, it is the network’s reigning business model as well. The faster we surf across the Web—the more links we click and pages we view—the more opportunities Google and other companies gain to collect information about us and to feed us advertisements. Most of the proprietors of the commercial Internet have a financial stake in collecting the crumbs of data we leave behind as we flit from link to link—the more crumbs, the better. The last thing these companies want is to encourage leisurely reading or slow, concentrated thought. It’s in their economic interest to drive us to distraction.

Unfortunately, I feel Mr Carr may have a point here, though as the methods for measuring behavior evolve (for example, by being able to factor in things like time spent looking at a page, rather than just clicks), this incentive may change (online publishers of long-form video, for example, absolutely want the user to sit in front of the same page for 20 minutes). But I don't have time to analyze it further - my inbox is filling up.

Online Advertising Business 101, Part I - The Online Advertising Value Chain

When you spend as much time as I do examining the workings of the online ad industry, it's easy to forget that, to many people, it really is pretty opaque. Not only is it characterized by some of the most complex and scalable technology in the world, but it also has its own, pretty unique, economic model to boot.

I've lost track of the number of times I've been asked by people, even super-smart colleagues from within Microsoft, "so, how does the online ad industry actually work?" So I thought I would attempt to provide a bit of a primer through the medium of this blog. Who knows, maybe someone will read it and offer me a book deal ;-)

In this first installment, I'm going to take a look at what I call the online advertising value chain:

This is a simplistic view of the industry, but it does enable us to understand where the key players sit; on the demand side of the value chain, there are advertisers, and their agencies; and on the supply side, publishers, and ad networks (and/or ad exchanges).

 

What's the product?

Before I get onto the content of the boxes in the above diagram, though, we should be clear about what's in the arrows; that is, what's traded in this market? What's the actual product, here?

The answer is advertising inventory. There are no very good definitions of advertising inventory out their on the Internet (Dave Chaffey offers one of the better ones), so I offer my own definition:

Advertising inventory is the supply of opportunities to display advertising in a particular medium.

Most people would use the term "ad impression" instead of "opportunity to display" - the reason I haven't is because I don't like to offer a definition of a term which contains another term that you may need to go and look up. The most common definition of ad impression is this:

An ad impression is a single viewing of a single ad by a single individual.

(Another reason I didn't use it is because it fails to capture the increasing complexity in ad inventory as online advertising evolves. For example, if you're serving video ads, and the user watched half of your 30-second pre-roll ad, was that an ad impression?)

In our value chain above, it's the Publishers who are the creators of advertising inventory. By building websites or software apps or video games or e-mails which are seen by lots of people, and inserting ads into these environments, publishers create a constant stream of ad inventory which, of course, they are looking to sell to advertisers. Agencies and Networks merely help the process along.

Online ad inventory is a very interesting type of good (to use the economics term). It has an incredibly short shelf life (measured in milliseconds as a page loads), but its supply is only indirectly under the control of publishers; external factors (such as a very newsworthy event) can dramatically impact the amount of inventory that a publisher has to offer. As a result, inventory prediction is a major task for publishers; I'll be returning to this topic in a future installment.

Calculating ad inventory

Another useful way of understanding ad inventory is to look at a simple example of how it's calculated. Imagine a pretty straightforward website (this blog, for example), showing pretty simple ads, with no fancy auto-refresh stuff going on (i.e. once a page is loaded, the ads don't change, so for each page impression, you get one batch of ads). How much ad inventory is created?

The answer to this is dependent on two variables - the number of page impressions on the site, and the average number of ads per page. So, for example, if my blog generated a million page impressions per month (I wish), and had an average of 5 ads per page, then the total ad inventory (if you're just using a simple ad impression model) is 5 x 1m = 5m ad impressions per month.

 

The Players

Now that we understand what's being traded, let's take a brief look at the major players in the value chain, and then I'll let you get back to whatever it was you were doing before you started reading this post.

 

The Publisher

We've already covered this guy. He's the one with the site, or the game, or the mobile portal, who is creating ad inventory and wants to sell it to advertisers to provide income for his business. Publishers are interested in maximizing revenues, but also at minimizing risk - they hate to have unsold inventory (that is, ad space with no ads in it) so they employ a number of tactics to ensure that at least something gets shown in an ad unit that they can get a little money for.

Larger publishers have their own sales teams who maintain direct relationships with advertisers and their agencies, cutting deals for big blocks of advertising inventory over expensive lunches in chic Greenwich Village restaurants. But this model only works for big publishers selling to big advertisers. Small publishers can't afford to maintain their own sales force, and even if they did, they'd never get through the doors of Ford, or CapitalOne, because they don't have enough inventory to be of interest on their own account. So these guys sell their ad inventory through Ad Networks.

One other kind of publisher it's worth calling out here is the search engine - i.e. Google, Yahoo and Microsoft. These search engines are the creators of huge amounts of ad inventory that is sold directly to advertisers and agencies, as well as running significant ad networks (see below).

 

The Ad Network

Ad Networks are essentially outsourced sales houses for publisher inventory. An ad network strikes deals with lots of publishers for their inventory and then aggregates this inventory and sells it on to advertisers and agencies. There are over 300 ad networks in existence today - a breathtakingly large number which is sure to fall soon.

An ad network's value proposition to publishers is that it can sell inventory that the publisher can't sell itself - either because the publisher is small (and so doesn't have its own sales force), or, in the case of larger publishers, the inventory is of too low-value to merit direct selling. This kind of inventory is called remnant inventory.

The network's value to an advertiser is that the advertiser can appear on lots of sites across the Internet (potentially thousands) without having to establish direct relationships with those publishers individually.

At bottom, the Ad Network business model is to buy inventory cheaply and sell it on at a higher price. There are a variety of ways of doing this, some of which I've covered before. One of the most promising is to add value to the ad inventory by adding targeting data (so that the impression can be sold for a higher price). I'll cover this in a future installment.

Networks come in all shapes and sizes. There are 'premium' networks which work with remnant inventory for large publishers; there are vertical networks which focus on a particular industry or technology (such as video); and, at the bottom end, there are contextual networks which provide an auction-based marketplace for selling keyword-based ads on small sites. You may have heard of the #1 network in this space - it's called Google AdSense.

 

The Advertiser

Advertisers also come in all shapes and sizes, of course. The big name advertisers - the folk we've all heard of - will have significant internal marketing departments, and will also likely retain the services of an agency to help them manage their marketing. Their marketing objectives will likely be a mix of brand marketing (raising general awareness) and direct response marketing (getting someone to actually buy something online now).

Smaller online advertisers are almost always focused on direct-response - getting someone to click and buy, or possibly call up. By and large, these folk can't afford to retain an agency to do their marketing for them, so they tend to go straight to certain ad networks or publishers to buy their ads. Again, the #1 in this space is our friend Google, with AdWords (the advertiser-facing side of the AdSense network).

Advertisers are motivated by getting the best ROI on their ad investment; but amongst larger advertisers some other curious motivations creep in, like wanting to make sure that a committed ad budget for a quarter actually gets spent (so that budget isn't cut the following quarter). This drives the behavior of ad agencies, to an extent.

 

The Agency

Last but by no means least, the media agency is an essential intermediary in the advertising value chain. Ad agencies usually do one of two things (or both, such as is the case with our own Avenue A|Razorfish): they create ads (anything from designing an animated banner to filming a 30-second TV ad) - known as the creative business - and they buy the media (i.e. the ad inventory) to display the ads (known as the media business). Whilst the creative side is cooler, the part of ad agencies that is relevant here is the media business.

A media agency, then, is one that buys media on behalf of its advertiser client. The advertiser typically says "I have x million dollars this quarter for online, and this campaign I want to run. Buy me the best media to reach my target audience". It's then the media agency's job to plan a media buy that will deliver the best return for the advertiser.

At the small-business end of the spectrum, the 'media agency' morphs into small SEM (Search Engine Marketing) shops who are good at buying Google AdWords, and maybe have some SEO (Search Engine Optimization) skills to boot to boost a company's natural search rankings.

Media agencies' motivation is driven by getting as much media under their control as possible, since they're paid (particularly at the high-end) with a cut (usually something like 15%) of the advertiser's media budget. They also don't want to under-spend on the budget they've been given, as this can annoy their client (see above).

Media buying is a manual, labor-intensive process right now, and one I'll come back to. Improvements to technology will mean that agencies (especially larger ones) will have to do some pretty fancy footwork to continue to add value for their advertiser clients.

 

That's it for now. in future installments, I shall be looking at the key players in a bit more detail, and looking at some of the interesting economics which underpin the industry. In the meantime, if you have a comment, or something you'd like me to cover, leave a comment.

[Update 6/3/08: A little more info on Ad Networks added]

Online Advertising Business 101 - Index of all posts

Advertising on the BBC

As anyone who has spent any time in the UK will know, the BBC is a safe haven from that most grubby of industries, advertising. It's enshrined in the BBC's charter. But the BBC's ban on advertising only applies to the UK (where anyone who owns a TV has to pay a $300 licence fee every year, which funds the BBC). Here in the US, the BBC is free to licence its programmes to commercial stations, and also runs BBC America, which carries ads.

But now, following a recent site redesign, some pretty substantial ads can be seen on the BBC homepage itself - at least, if you're outside the UK:

The BBC is within its rights to do this, of course, but it does raise some interesting questions. The content on the BBC website is created using licence fee payers' money, a distortion of the UK news website market which has been controversial for as long as the site's existed (the excellent BBC News site competes with the likes of the Guardian, Times, Independent and so on in the UK).

With the traditional newspaper publishers (and commercial TV stations, to a lesser extent) starting to really feel the pinch of the Internet (with advertising budgets fleeing to Google and other places), sites like The Guardian's are making efforts to expand overseas in an attempt to tap into the US advertising market. So the BBC's entry into these markets has not been greeted with much enthusiasm.

As a Brit, I have a huge soft spot for the BBC; the licence fee has subsidized some great TV that wouldn't have otherwise been made; but I wonder how long its its funding mechanism can continue, as TV and online increasingly merge together, and national boundaries for content consumption are blurred. At the very least, it seems hypocritical to me that I am subsidizing the BBC through being served ads on the one hand, yet unable to watch any of the video content delivered through the iPlayer (the BBC's catch-up service) on the other, because I'm outside the UK.