PR Measurement Needs to Look Up Not Down

Better metrics and frameworks aren’t the primary solution to public relations measurement. We need to ask better questions first.

In this post  (originally posted on the blog of Stephen Waddington who often covers PR Measurement I talk to Jon Sellors, a client-side head of communications and Stephen Ashcroft, a procurement expert about why public relations measurement is too focused on dashboards and, with their help, I set out a vision of what needs to change if measurement is to demonstrate the strategic value of public relations and a strategy to achieve this.


As so often, it started with a tweet.

It’s time for public relations measurement to put down the spreadsheet and look up and accept that its focus on frameworks and dashboards is not working.

In making this claim and in setting out the way I think that measurement has to change direction to be more valuable and relevant, I have sought the client perspective from Jon Sellors [@jonsellors], head of communications at LV; and Steve Ashcroft [@thinkprocure], an expert in procurement.

I do this not only because they have valuable opinions, but because the solution to PR’s measurement problem has to be one where both buyers and sellers of public relations recognise that measurement is not a negotiating tool towards higher or lower fees, but that all parties have common ground in demonstrating the strategic value of public relations.

Thesis 1: measurement is not an end in itself

Public relations measurement is and always has been focused on data points at the end of a process. It has become more sophisticated and the data points have become more diverse and numerous, but it is ultimately what engineers call an “end of pipeline solution” – trying to fit better filters to the outflow, rather than looking up to work out why this stuff coming down the pipeline is the colour it is in the first place. The opportunity is to be better at looking upstream and ask ‘why’ questions, not ‘what’ questions.

Jon: I Totally agree that public relations measurement has been and is still focused on the end of the pipeline. Evaluation reports have certainly become lengthier covering more data points but I’d question whether they are more sophisticated.

My frustration with media evaluation suppliers has not changed over the years – they are very good at producing a pack of pretty graphs but is there anything actionable? I’m ashamed to admit that I can’t think of a time where I’ve changed strategy or plan based on intelligence I’ve received from my media evaluation. So why do I keep paying for what’s little more than an expensive comfort blanket? Probably because we’re told you must measure.

Thesis 2: Despite their flaws, AVEs persist for a reason and we are not asking why measure public relations not media relations

The situation, as evidenced by the latest spat over advertising value equivalent (AVE) (and the fact that after so many years they even merit a mention in the latest version of the Barcelona Principles) says to me we are long past the point where we should challenge the public relations measurement sector (the systems vendors and various measurement panels) to practice what they preach and measure their effectiveness in trying to get buyers of public relations services to adopt the Barcelona Principles – my hunch (backed up by judging a big public relations awards scheme for two years and the briefs I see) is that they simply don’t appear on the radar of clients and are not really observed in practice. 

Jon: Yes again. The AVE debate has raged for many years but the fact that it still exists shows something more fundamental is wrong.

We can’t keep saying they are not appropriate and expecting the situation to change. As well as challenging the industry to practice what it preaches (and I saw an AMEC member and potential new supplier recently and AVEs were still included in the sample charts they gave me) I wonder if we need to challenge what is meant by the ‘public relations measurement industry’, it needs to move on from just analysing coverage.

My take from judging awards is that measurement is included as an afterthought and goals or targets were not set at the outset hence so many vague comments such as “the client was very happy” and “lots of coverage was achieved.”

Thesis 3: measurement frameworks don’t allow for the agile and iterative nature of campaigns

Frameworks and principles which can be boiled down to “establish the business goals and then measure success towards them” are fine as words on paper, but ignore some very serious obstacles in practice (this is not an exhaustive list):

  • What you describe in a brief and how I interpret that in terms of measurable results can be very different – phrases like “think outside the box” are highly subjective
  • “No plan survives contact with the enemy” (ie. a sceptical media/audience that cannot be bought, or their response reliably predicted)
  • Shit happens and long term goals are prone to getting chucked out when the next quarter’s figures look unattainable
  • A lot of briefs are plainly cut and paste jobs where several different people with different desired outcomes/agendas are all convinced they have given a precise instruction set and will have very different ideas about what success looks like.

Jon: I would add another element in here. I think public relations agencies and measurement suppliers need to help in-house teams educate boards/exec teams about the role of communications and therefore what should be measured and reported back to them.

If a senior stakeholder is used to someone giving him/her a monthly AVE, reach or share of voice figure we need to effect change around the exec/board table first.

I don’t think that the public relations measurement industry can exist in isolation. People don’t consume media or form impressions of brands in isolation. Measurement has to be much more rounded and sophisticated. I’m increasingly looking at overall measurement of reputation rather than analysing coverage.

While there has been much talk over several years about the blurring of boundaries between public relations and marketing, driven particularly by the growth of digital and social channels, I think the new blurring is with public affairs and corporate responsibility. Hence why I’m now more interested in measuring reputation and doing this across various stakeholder groups.

So, what’s the solution?

We should be bold and completely change the public relations measurement debate from the end of pipeline pursuit of ‘better metrics‘ and theoretical frameworks, to the pursuit of ‘better questions to define the problem to be solved, at the outset’.

 That would have three strands:

  • buyers and vendors of public relations services collaborating on best practice for RFPs and ongoing campaign management in writing a brief in which objectives are described in precise, unambiguous language and written in a way that anticipates both variations and measurement.
  • collaboration on standard models of payments and incentives which force buyer and vendor to be precise about communications objectives and precise about how a midstream deviation from objectives is agreed and incorporated into the risk/reward in the contract
  • buyers and vendors collaborating with procurement specialists to agree how much weight is attached to cost in a contract and how much to measurable outcomes

Sounds nice, but it’s not easy so how could the industry develop a more collaborative, rather than “go on then, prove it” way of thinking about measurement.

Here’s where the opinion of procurement expert Steve Ashcroft [@thinkprocure], who blogs at ThinkProcure, comes in and his response to the challenge of how do we deliver against these three strands  is a set of questions for both buyer and vendor of PR services which would go a long way to establishing measurement and collaborative best practice – leading up to the big, abstract question, posed at the end.

Steve:  If I was advising a buyer keen to explore a more collaborative procurement of PR services, here’s where I’d start. A little bit of humility by asking questions and listening – carefully – would be a good starting point to establish co-operation and, heavens, maybe even trust.

Q1. Do you the client know what you are buying? Admit you are not (always) a technical expert – get the PR services provider to explain their value in business terms.

Q2. Are the responsibilities clear (who, what, why and when)? Let’s be clear on obligations and liabilities (note you, the Buyer might have some).

Q3. Has the PR services provider explained how the deliverables are going to be achieved? A method statement or plan (you did ask for one, right?).

Q4. What does the PR services provider need from you? Dependency on the Buyer can we a useful excuse for failure – by some PR services providers. Clarity is key.

Q5. Are deliverables linked to acceptance prior to payment? You need to define and jointly agree what is acceptance (agreed prior to contract award).

Q6. What happens if things go wrong? We always ask the (big-spend, strategic) PR services provider for their risk register (they have got one, right?).

Q7. Are responsibilities to report progress clearly stated (no surprises)? Ask the PR services provider to define Management Information in their Proposal. (I am aware of the Barcelona Principles, let’s see if your PR services provider can articulate how they will be reported (and delivered against) on your project.

Q8. How is the PR services provider’s performance monitored? Is everything on track to ensure delivery? How do you know?

Q9. What will success look like? (the quantitative values as well as the qualitative values (key messages; mindshare; influencers, customer or analyst credibility). Clicks, conversions and sales are value, of course! Can the public relations service provider articulate success metrics? Are these clear in the contract?

Q10. If it is not clear to you, how can it be clear to the public relations services provider? (Admittedly a question for you, the Buyer more than the public relations services provider).

What is ‘it’ in Q10? It is the complete procurement – the brief, the metrics, pricing, contracts obligations and liabilities – the whole relationship in fact.

Misaligned expectations are the number one cause of commercial disputes. Buyers – try out the questions – and for public relations service providers on the other side, are you sure you can evidence answers convincingly?

Moving on, and up

Goal congruence between buyer and vendor through a collaborative, trusting relationship with evidence-based performance visibility is not even remotely easy, so why bother?

The prize is enormous and a shared one for buyers, vendors and procurement people – clarity, standardised practice to enable valid comparisons and an absolute focus on addressing a business’ true objectives in a way that is recognisable to the board.

The measurement debate has been useful and valuable, but we need to move on and move up.

If you think that’s worth exploring then let me know and let’s get some smart people in a room. You can find me @NigelSarbutts or of course, on here.

Let me know what you think.

When Should a PR Person Step in to End an Interview?

Consumer electronics company VTech has provided a useful case study in the importance of preparing for a media interview and the decisions to make on whether to be interviewed at all.

An interview between the BBC’s technology correspondent Rory Cellan-Jones and a VTech senior brand manager at a trade show went a bit sideways when he asked whether the company could recover trust with consumers after serious vulnerabilities in the company’s data security had been revealed.

Cellan-Jones asked a direct but straightforward question about consumer trust only for the off-camera PR person to step in to halt the interview and offer her opinion that this was not the story of the day.

It’s awkward and you can see it here

Via Twitter I asked if a line of questions had been pre-agreed for the interview.

Here’s the exchange

My point is that VTech had a significant, current and well-reported issue associated with their brand and the probability of a senior BBC reporter asking about it was high.

In this situation there are three possible paths, with different outcomes:

Agree what is and what isn’t going to be covered in the interview. If the interviewer breaks that agreement and this new line of questioning is likely to be damaging then you need to weigh up whether you are going to intervene on the basis of what had been agreed. (It’s good to decide whether you yourself are up to doing that before the cameras roll…)  It’s risky but it’s hard for a respectable media outlet to run footage with someone politely explaining that they have broken a promise. If the story is about serious matter of public interest, then forget it, assume everything is up for grabs and the chances of agreeing terms are much more slight.

If you can’t agree to a list of topics for the interview (and especially if you know that there is an elephant in the room) then you have to field someone for the interview who is able to respond if the problem is referred to. I wonder if a brand manager is really authorised to comment to the BBC about an issue affecting broad corporate reputation. From the footage, it seems not.

If you can’t agree topics and you can’t field someone with the authority and ability to handle a tough, big question then you either decline or you hope that no-one notices the elephant.

It’s hard to decline the chance of a top tier media interview, but as this shows, they come with risks.

For the avoidance of doubt whilst I asked Mr Cellan-Jones for permission to quote our exchange, he has not contributed to or seen this piece before publication and this is my personal opinion.

Lucy Kellaway’s Response to HP is Important For PR

(a repost from my LinkedIn profile)

PR consultants should be outraged at Lucy Kellaway’s attack on one of their own, right?

She has gone public with what was probably intended as a private conversation and pretty deliberately rubbed someone’s nose in it.

Is that fair?

Of course it is. If the HP’s PR guy didn’t consider the risk that his approach would be reported, then he is naïve.

What I’m hacked off about is the reaction from journalists which has been virtual fist bumps and air punching, as if this is sticking one on the PR people (with its unspoken “the bastards” undertone).

Here is level headed Rory Cellan-Jones from the BBC.


Well Rory and others, PR people are pleased too.

I celebrate Kellaway’s column and I celebrate the FT both for telling HP what it can do with its implied threat and then give Kellaway free rein to write about it.

I’ve read Kellaway’s original column (to which HP has reacted so badly) and to my mind it was funny, sharp and true. HP’s CEO, while her comments at Davos opened the piece, didn’t come out of it half as bad as some of the others quoted.

The line between advertising and editorial is sacred and PR people should fight for it to remain so.

The Telegraph has been accused of blurring the line in recent times and you can draw your own conclusions about whether this has any bearing on how its readership compares to the FT.

The separation of advertising and editorial is the datum point for everything that we in PR and communications do and if advertisers and publishers start confusing where it sits to suit some short term need or to soothe some CEO’s ego, then everyone loses, PR people very much included.

PR at its heart is all about negotiating with a journalist or editor about the intrinsic value of a story. Make that a transaction and everyone is compromised and you cease to be able to value a story on its own merits.

I don’t want that and I don’t think clients would want a world where that is the case.

Bravo Lucy Kellaway, bravo the FT and to HP…you’ll get over it.

Marketing Procurement, Let’s Collaborate

This a re-post of a piece I was asked to write by Steve Ashcroft, a prominent procurement expert.

He asked me to develop an idea I had that marketing agencies and procurement professionals face similar challenges in proving our value at executive level and that instead of sharing our thoughts on how to overcome this, we are wary of each other.

This doesn’t seem to me like a smart way to solve a problem.

Here’s the piece…..

“I’ll be honest. I was pleased to read that PepsiCo has shut down its marketing procurement function and is giving the responsibility to its brand management teams.

Not ‘ding, dong the witch is dead’ rejoicing but I think it’s right that these decisions are owned by those accountable for the effectiveness of campaigns.

I also firmly believe that yes, marketing is a special case for the 10 reasons explored in Gerry Preece’s short book ‘Buying for Less – how to avoid the marketing procurement dilemma’, which can be summarised as:

‘Marketing is an investment not a cost and there are too many variables within the vendor set and unmeasurables in outcomes to make marketing procurement any more a precise, predictable activity than marketing itself.’

That’s not an argument to say don’t do it or that marketing deserves a free ride.

It’s an argument to recognise the limits of the possible and in the absence of precise, fair methods of evaluating the effectiveness of an activity, filling that void with a focus on cost alone is unwise and counter-productive, not just for both parties in the negotiation, but for the credibility of Procurement itself.

I want to offer a way forward which both protects the value at risk in a developing a marketing strategy and also does something for the status of the people around the table too.

The origins of this piece was Steve’s own observations recently that procurement is struggling to make a credible impression in the C-suite.

This, for me at least, was a striking thing to read for two reasons.

Firstly that this is something I read almost every week in the marketing press – why are there so few marketing directors on executive boards etc.? So it struck a chord. The second reason was that I’d kind of assumed that Procurement didn’t have any worries in this department.

Marketers generally think of Procurement as the shock troops of the FD’s team with a direct line in to the boardroom.

So it’s good to know that existential angst is not the marketer’s sole preserve…we’re not alone in the universe.

Let’s then ask why we have this failure in common, the answer to which is encapsulated in Preece’s book.

Measuring marketing effectiveness is really bloody hard and I can understand that trying to pick your way through a blizzard of vendors with highly variable, subjective qualities must be extremely challenging or even impossible for Procurement professionals.

In my own field of expertise, public relations, an argument has raged for as long as I’ve been in the industry (about 30 years).

How do you measure PR?

For years the industry fiddled about with a measure called AVE or advertising value equivalent. It’s the idea that the efforts of the PR team would produce a piece of media coverage that can be measured with a ruler (stop laughing) and then compared to how much it would have cost to buy that space as an advertisement.

Where to start on the idiocy of that idea….it’s like a restaurant critic evaluating a meal by the number of calories per £ they ate.

It’s still in use today despite the efforts of the PR industry to banish it as absurd and outdated.

The reason it hangs around like a bad smell is that nobody has come up with a simple, universal measure. The alternative (to map outcomes of work to precise and unbending campaign objectives) is demanding, expensive and requires a level of sophistication that is not found in the day to day management of most marketing campaigns which are exposed to changes of direction by the day if not the hour and objectives become a highly malleable concept.

Instead of doing the thing that is useful but hard, people fall back on the useless but easy and the hardest part of marketing is setting the objectives and the strategy.

I still read briefs from experienced marketing people working in big, well organised companies that say things like ‘make people visit our website’.

When was the last time anyone made you visit a website?

This sort of idiotic instruction set or spec is both Marketing’s problem and Procurement’s and will remain the principal barrier to the C-suite.

When you have a common problem, you need a common solution, which is for vendors to sit down with Procurement people and try to agree ways of evaluating marketing outcomes that both parties can support.

That is idealistic and a more practical objective would be to agree ways of describing desired marketing outcomes, in other words, work on the language of the instruction or specification to drive out uncertainty and make it easier to link objectives to outcomes.

Phrases like ‘raise brand awareness’ or ‘think outside the box’ are uselessly vague or subjective and I would recommend David Owens book ‘Creative People Must be Stopped’ for a clear solution of how to cut this challenge down to size.

He explains why and how imprecise language constrains innovation and the evaluation of risk in areas of activity like marketing because it fails to capture the constraints on innovation and more importantly he shows how to solve the problem.


We know what the problem is.

We have common cause in solving it.

We have the tools to address the problem.

Who will help me take this forward? I am genuinely interested to hear from Procurement professionals who would like to discuss this further.

Nigel Sarbutts, from helps companies get more out of their marketing investment by challenging boards to define and communicate their objectives more clearly and marketing teams to improve the quality of their response.

In Praise of Volkswagen – a PR Masterclass

In PR terms, Volkswagen might be giving us a masterclass in crisis response strategy.

The statement by Michael Horn the CEO of Volkswagen USA is in my opinion as good as it gets:

“Our company was dishonest. We have totally screwed up. We must fix the cars to prevent this from ever happening again and we have to make this right. This kind of behaviour is totally inconsistent with our qualities. We are committed to do what must be done and to begin to restore your trust. We will pay what we have to pay.”

From a technical point of view it is very clever because it uses short sentences that cannot be misinterpreted and each one deals with a separate dimension of the scandal – ethical and technical and it deals with actions and financial commitments. Anyone reporting it is almost obliged to quote it in full and in so doing VW gets its message across unfiltered. That, of itself, is solid gold in a crisis.

It’s the strategic aspect that I find admirable.

It is obviously the opposite of the mealy mouthed pseudo apology that has felt the dead hand of a lawyer and that is why it is brilliant from two strategic points of view.

Firstly, it makes it very hard for this story to get any worse for VW so it gives them, as far as they can possibly hope at this point, a solid foundation, not shifting sands. The only way is up.

Secondly and this is where I think they might be being very clever indeed it sets the standard for other auto manufacturers who might be about to find themselves in the same position.

Any other auto brand that is engulfed by this scandal which does not make a similarly comprehensive and unambiguous mea culpa with its commitment to swift financial action, is going to find themself the attention of the media as story unfolds and expands to become one of not only wrongdoing, but corporate cowardice.

It seems unlikely that anyone is going to beat VW for directness and an apparent willingness to deal with this problem and confront their own shortcomings and that gives them a strong hand when the immediate sense of scandal has cleared.

The world isn’t going to stop buying cars, even diesels, and if most if not all brands in the category are swept up in this scandal, then the one that makes the most authentic and fully resourced commitment to solving the problems of its failures will be the one that comes out of it first and can begin re-building its most valuable asset. That means the one best able to capture market share from its category rivals.

It’s a big bet, but there is no halfway house here.

Bravo Volkswagen