Welcome to the Vortex of (Metrics) Hell
Josh Chasin of comScore opened the panel by defining “maelstrom” for the audience. I’ll let you go off and look up your favorite definition, but suffice it to say that Josh welcomed us all to the Vortex of Hell. It was a great mix for the panel with representatives from Panel based companies (Josh from comScore and Jon Gibs, VP, Media Analytics, Nielsen Online) along with top industry analyst Young-Bean Song VP of Analytics & Atlas Institute and veteran Beth Uyenco, Global Research Director, Microsoft Digital Advertising Solutions. Josh opened by defining a few questions for the panel to discuss and each panelist had a few minutes to talk to the audience.
While it’s nice for the panelists to get to present an introduction, I really don’t care for a set of charts to start a panel. That said, it was refreshing to hear Jon, the panel measurement representative come out and admit that panel data alone is not enough. He said that an integrated approach is best, taking the best of the panel data and server side data. In fact, his stated goal is to “move all measurement to integrated on next five years.”
Young-Bean gave a very silly but pointed example of a patron at a bar walking past a neon sign for Corona at a bar and ordering a Corona. An observer in the rafters could look at this and say that he ordered the Corona because he walked past the sign. This is no different than today’s logic of giving 100% of the credit of a sale to the last click. He noted that “when you go upstream you realize people are being reached over many channels. The question for now is how to take all other touch-points and use them in media planning and buying. We’re only using 5%-8% of touch-points now. The other 90%+ is being ignored.”
Beth wanted to let the audience know that she does not think that there is a need for a single “currency” of measurement. Based on her many years experience with brands and media she said that the “truth is that there are a variety of channels that all happen to be digital. They behave like offline, but we don’t have a single currency for all those others. Why have one for all kinds of digital? Different channels have different objectives.”
It does seem that there is a lot of baggage. Beth said that “we need to let go of established practices we’ve had for the last 15 years.” With so many options there are new ways of analyzing and collecting data to draw insights where we were hampered before by more limited tools. She talked about using data to understand where the behavioral trends are, but also cautioned that “we can’t continually slaughter lambs and virgins for every campaign we have to analyze.”
With TV, radio, and print, it is easy to “measure” because you have a single way to look at the data and there is not much to dig into. The Web is infinitely measurable which is why it is so great, and why it is such a pain to deal with. When you have so much data, there are many ways to interpret it. And when the bottom line is that a planner and agency will sit down and look at a report to make a decision you need to give them something to help them determine what to buy more of and where to buy less.
Young-Bean had an example of measuring the difference between search clickers who had been exposed to display ads and those who had not. There was a 22% lift for those who had beed exposed to display. But that is not actionable. There were 12 different advertisers that saw a range of increases. In fact, one had zero lift and another 65%. He said that “you need to get down to the placement and site level” to really have something actionable.
One of the more interesting question from the audiencerelated to past failures of the industry to actually set up an integrated approach to measurement, specifically pointing to Project Apollo, which I actually reported on in 2005, as the most recent failure. The panel seemed to agree that Apollo was not supposed to be a holistic approach that would give a common currency, and that it instead was a test lab. Apollo was one approach to a fusion of data, but perhaps too extreme, trying to collect all information possible from a group of the same people. Not that the other extreme of getting all the shopping related data without looking at media exposure is any more useful. This is where fusion of multiple data streams can work to give results that are actionable and more relevant than we have now,
