Let’s set the scene a little first.
Consumer behaviour may be changing rapidly but traditional media still remain important.
Thinkbox data shows us that broadcaster TV accounts for 91% of our video advertising daily. Don’t forget that Ofcom figures put TV advertising revenues at £4.5bn in 2019 and while that will have declined due to Covid last year, it is bouncing back. When you add in BBC Licence Fee income of £3.5bn in 2019/20, an enormous amount rests on having accurate ratings data.
Invisible and important
RSMB is a contractor to both BARB and RAJAR and our responsibilities include ensuring that BARB TV ratings are correctly weighted and checking that people on the BARB panel are doing what they should, so that the ratings reflect actual behaviour.
The work we do for BARB and RAJAR is largely invisible to end users, as it should be. But if it went wrong, the ripple effect on the industry would be significant and expensive. When it comes to audience measurement, the devil really is in the detail.
Reassuringly boring and proud of it
How have things changed over the years?
Our view on complexity has changed over time. First we had four terrestrial channels in TV measurement, then tens of analogue satellite and cable channels and then hundreds of digital channels. Back then, increasing panel sizes was seen as the solution. But actually you have to quadruple the sample size to halve the sampling error so the benefits are fairly limited in a landscape of hundreds of channels.
Now we’re looking at incorporating return-path data from set-top boxes, or server logs or tagging VoD content so it can be identified on the panel. Some of the data is at the household rather than individual level, so you have to add a demographic profile to it.
These are highly complex data science tasks but they’re necessary to ensure that measurement reflects changing patterns of consumption, otherwise that viewing won’t be recorded at all.
RSMB spend a lot of time evaluating and developing these statistical methodologies. We’ve been described as reassuringly boring. When it comes to the stability of ratings, that’s probably how it should be.
Cross-media solutions exist but who can you trust?
We’re in a transitory phase and a variety of approaches can co-exist, or, even better, can be combined to produce improved solutions.
RSMB has delivered industry initiatives such as the IPA’s Touchpoints Media Planner (which integrates the main currencies to allow cross-media planning) and Sky’s CFlight, a sophisticated means of measuring campaign reach and frequency across its linear and VOD offerings. They sit in a wider landscape of industry solutions which include BARB’s deployment of Kantar’s focal meter to measure streaming and the BBC’s Compass cross-media survey.
It seems everyone has their own solution to the problem of cross media planning and evaluation. But to what extent can they be trusted?
In the absence of a single industry solution, there are a number of audience approaches out there; some are better than others.
Anyone relying on a cross-media model for planning or campaign performance should satisfy themselves that it has been properly designed, evaluated and implemented. Otherwise they may as well be planning randomly. Models can be tested to establish their “regression to the mean”. That is, the extent to which the model performs better than randomly matching the datasets together.
RSMB rigorously test our own integrations and, as specialists, we’re often approached to independently validate others’ models”. ISBA’s Project Origin, for example, has asked us to evaluate the World Federation of Advertisers’ Cross-Media Measurement Framework.
There isn’t a perfect model right now and there may never be. It’s fine to build a model, test it, be transparent about its weaknesses and strive to continuously improve it as better data becomes available, as long as the underlying methodology is valid.
Apples and Pears
While a good model may perform well and deliver a good indication of cross-media exposure, there are other factors that need to be taken into account, such as equivalence, attention and impact.
It’s important not to look at incremental reach delivered by a 3-second ad over a traditional 30-second spot and assume they have the same impact. Models can give the impression of equivalence when that is far from the case.
The future
I joined the company at a pivotal moment: this is an exciting time to be in audience research. The media industry will continue to evolve, with fragmentation posing even greater challenges to existing currencies and research methodologies. And together, we will need to be increasingly creative in how we meet these challenges and how we make best use of the opportunities they provide.
What will audience measurement look like in 10 years’ time? I believe the industry will have agreed on a common cross-media solution by then. The building blocks are likely to be many (but not all) of the industry currencies we use today, as well as return path data and server logs plus a cross-media survey to act as a hub. Data will be integrated into that hub using open and industry-agreed standards.
Ensuring accuracy of the building blocks will continue to be as important as the model used to combine them. There will be challenges in getting the industry to agree methodology and fund it, but the direction of travel is clear. I would encourage the industry to invest properly in audience research given so much depends on it.
RSMB has been responsible for many of the UK’s innovations in audience measurement in the 30 years we’ve existed. From combining BARB and TGI data in the 90s and improving the BARB methodology by making sub audience ratings more robust, to integrating media currencies in the Touchpoints Planner right through to our work with CFlight today.
One year into this role and looking forward, I believe the industry can rely on RSMB to both develop robust cross-media, cross-platform, consumer-centric measurement solutions and support industry currencies to meet the challenges of rapidly changing media consumption.
