<< Back to Learning Resources
Approx Time

Chief Executive at international strategic consultancy Rouser, JP Hanson talks about the pitfalls that organisations trying to measure marketing effectiveness face, and argues for a strategic and structured approach to avoid them.

In The Kingdom of God is Within You, Russian literary giant Leo Tolstoy wrote: “the most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of a doubt, what is laid before him.”

It is nigh on impossible, for example, to explain to a manager that the notion that employee satisfaction drives performance is a halo effect (performance more likely drives employee satisfaction – nobody wants to work for a loser). To paraphrase Upton Sinclair , getting someone to understand something when their salary depends upon them not understanding it is difficult.

“There are financial incentives for not embracing external interpretations of what effectiveness is”

Similarly, many marketers claim to know, without a shadow of a doubt, what marketing effectiveness entails. Though experts within the field are quick to label this a Dunning-Kruger effect, the fact remains that there isn’t a universal definition of the term. This means that we end up with a situation close to that which Sinclair described; there are financial incentives for not embracing external interpretations of what effectiveness is or certain measurements of it. To illustrate, it stands to reason that someone who has an incentive to buy or sell digital media is not only likely to promote a definition of effectiveness that is close to efficiency, they are also likely to promote multi-touch attribution models.

The effectiveness perception gap

As a result, brands often find themselves unsure of the best course of action. While readers of this space are familiar with the important work of Les Binet and Peter Field, many people are not. Where there is a lack of defined verbiage, there is room for perception to outmanoeuvre reality.

Related content – watch Binet and Field present their latest research at EffWeek 2018

According to the 2018 Nielsen CMO report, social media and search were perceived as having the highest effectiveness of all channels, with 69% of respondents (in each case) claiming they were either “extremely” or “very” effective. A mere 43% considered TV to be effective.

In reality, TV is the most effective media channel on the planet by quite some distance, at least by the definition of effectiveness that many, including Binet and Field, employ. But herein also lies the problem. If we do not agree on what we mean by effectiveness, how can we agree on what communications options, or tactics in general, are most effective?

A strategic framework to bridge the gap

Any marketing effectiveness effort, consequently, must start with the establishment of a framework. Only when we have defined what we mean, can we identify and agree upon what is required to measure and forecast.

“Marketing measurement in practice is often about validating work to people who don’t believe in it”

The definition should be tailored to the individual company and relevant not only to marketing but also the wider business strategy. Alignment acts as a step toward management buy-in; marketing measurement in practice is often about validating work to people who don’t believe in it. Getting the finance department on board throughout the process, and ensuring that the marketing effectiveness targets set are aligned with the overall needs, will build organisational credibility.

EffWeek 2019
Creativity & Effectiveness Reimagined
Download presentations

Additionally, an established effectiveness framework will help prevent silo formations. Organizations are chain-linked. When each link is managed separately, not only does the company tend to get stuck in a low-effectiveness state, but there are fewer incentives for managers to invest resources in making their link better if other managers are not. Some management teams try to counteract this by awarding budget relative to performance, but end up invoking Goodhart’s Law. E-mail marketing teams, for example, might spam customers to get more conversions and, thereby, more budget. A framework prevents such scenarios, and for organisations that are active across international markets, it improves compliance. Holding local branches to the same established process creates far less risk that they ignore global directives.

Definition of a benchmark with profit in mind

The next step is to identify a benchmark metric that is appropriate for the strategic intent. The goal of effectiveness is to create a clear understanding of what is and isn’t happening, and a leading KPI allows for more accurate testing. Measuring too many things at once is not only expensive but often leads to contradicting data and analysis paralysis. Of course, proxies and alternative models can be used to solidify the benchmark, but it is important not to use data the way a drunk uses a lamppost – for support rather than illumination.

It is imperative that the effectiveness benchmark metric considers the long-term, which means focusing on things other than ROMI. Any marketer looking to spend company money will have return demands thrust upon them whether they like it or not, but measuring effectiveness through efficiency leads to numerous false conclusions and a dangerous bias towards the short-term.

“It is important not to use data the way a drunk uses a lamppost – for support rather than illumination”

In IPA research, very large market share effects were found in only 3% of short-term cased analysed. For cases exceeding 30 months, it was 38%. ROMI-focused activities also often target consumers with established affiliations to the brand and imminent purchase intentions at the cost of brand growth, long-term base sales and margins. Unsurprisingly, long-term cases (6 months or more) drove an eye-opening 460% more market share growth than short-term cases did. In other words, not only is there a clear incompatibility between maximising efficiency and maximising effectiveness, by measuring the latter using the former one gravely undermines the long-term profitability potential of the brand.

And, when it all comes down to it, the most important thing for marketing is to drive profit. Then again, I would say that – I’m a business-first strategist who works with effectiveness frameworks and whose salary, consequently, depends on it being true.

Explore our learning and resources for more help understanding how marketing effectiveness can be implemented in your organisation

“Just what the industry needs, great collaboration between clients and agencies on the topics that drive business growth.”

Bridget Angear, Joint Chief Strategy Officer at AMV BBDO

“It’s great to see the IPA in the UK bring the whole industry and particularly the trade bodies together to focus on effectiveness. This new Marketing Effectiveness initiative will enable people across the industry to work together to build on best practice.”

David Wheldon, Chief Marketing Officer, RBS

“Effectiveness is a team sport, so it was great to see the industry in the widest sense, come together. In an increasingly diverse and fragmented world, only by using all parts of the brain will we solve effectiveness challenges and design our campaigns to deliver short and long term value. That’s why what happens next is important – if the IPA can help facilitate progress on this with a long-term initiative around Marketing Effectiveness, we’ll definitely crack it.”

Bart Michels, Global CEO Kantar Added Value and Country Leader Kantar UK

“The time spent at #EffWeek was extraordinarily effective. It was great to hear the diverse views from all areas of the industry. All tied together with the common themes of accountability and effectiveness.”

Andrew Canter, Global CEO, BCMA

“It has been a privilege to be part of the inaugural Effectiveness Week. The agenda is one which we at O2 UK feel passionately about. To see and hear perspectives across the industry demonstrates how the breadth of marketing effectiveness is increasingly being valued within businesses. Data, insight, social, customer experience, test and learn, ROI, these are all fundamentals and were covered expansively at the event”.

Sandra Fazackerley, Marketing & Consumer, Telefónica UK Limited

“The full week of effectiveness events brought into clear focus the need for marketers to use data and insight to achieve the key business objectives of growth and profits. Marketers today are in a better position to quantify their knowledge of customers and measure the ability of investments in marketing to increase brand and shareholder value.”

Chris Combemale, Group CEO, DMA