Build and communicate evidence of impact: Notes from PINF's Zoom call
The third in the series of Zoom conversations on the ideas coming out of the Independent News Forum was held last week, with participants discussing efforts to build and communicate evidence of impact. Joe Mitchell, PINF’s head of impact, writes up the notes and next steps from the call.
Previously, on PINF’s blog:
We held an Independent News Forum in October 2022 in Leeds – recap here.
That forum led to five ideas for boosting the independent sector: defining new models of journalism; creating a PR campaign for independent news; building and communicating evidence of impact; building a distributed support network; and encouraging deep community engagement.
Throughout Jan and Feb, we’re hosting an online call on each of those five ideas.
You can see the notes from the calls on new models of journalism and on a PR campaign for indie news in previous blogposts.
In the third call of the series, around 15 people, including publishers and academics, met up to talk about impact. Thanks to everyone who came along. The group included people who were relatively new to the idea of measuring impact, as well as people who were already maintaining impact records and reporting on this to funders. And plenty in between.
‘We talk about ‘impact’ all the time, but what does it actually mean?’
Lucas Batt of The Bristol Cable gave an overview of where the group in Leeds had got to: from thinking about the kinds of impact we might be looking for (at a micro/individual level to a macro/governmental level); to the importance of predicting your impact, rather than looking for post-hoc explanations; from developing theories of change to looking at the tools for gathering evidence and whether these could be shared; and from thinking about the audience for the evidence and to how to present the evidence we find.
We then broke into smaller and larger discussions that covered a range of topics - I've bunched these below:
The why and the who?
Why does measuring impact matter? Who is the audience for the evidence we collect? The answers to these questions will differ per news outlet – and as such all outlets might end up needing to measure different things. You can’t measure everything – so the first step in any work here is to understand what you’re looking to find out and for whom.
Are you looking to check that you’re doing the best for your readers? Or is the impact you really care about on your volunteers? Are you looking to impress upon policymakers the importance of this work? Are you committed to showing a funder that their money has made a difference to the local community?
And what are the risks of these approaches? You might spend a lot of time/money proving impact to funders, but is that truly serving your readers? Can a balance be struck? Similarly, there’s a risk of overdoing it or measuring the wrong thing. What data do you definitely need? And what is actually measurable? What if what matters is really hard to measure?
Once you’ve established why you need to collect xyz data, and worked out how to do it, the communication of that evidence is also important. It's unlikely that a funder or policymaker is going to be able to read vast reams of data and get their heads around it all – what are the key themes you want to get across?
Examples of impact
At a macro or national level, PINF is always on the lookout for evidence about the societal or economic impact of news outlets, of which a good deal exists in the US (e.g. on democracy), but which is in short supply in the UK. PINF is also interested in the differences in news provision by local area in the UK – see our mapping project – and the group discussed how this would benefit from added content analysis to check that the claimed coverage of a title is truly matched by quality locally relevant content.
At the local level, the group surfaced a few examples of impact from some of the outlets in the room, such as having unsatisfactory councillors stand down, supporting civil society campaigns, and seeing ‘ripple effects’ of starting a community media project – inspiring new such projects outside the original area. There was also interest in content analysis to look at the tone of the news – one title was set up partly as a result of the negative tone around a place: several years later, had it led to a more positive tone in local media and social media?
Universities and research
Partnering with academic research expertise could be valuable across a range of impact projects, but academic funding is hard to get and a long time in the making. It was noted that often universities do have their own small pots of funding for local projects – so partnerships could make sense between news outlets and universities in the same place.
Next steps and potential roles for PINF and others
Over the course of the call, several ideas of next steps or facilitation roles were suggested.
A(n online) library of useful documents, such as case studies, templates or examples of reader surveys, impact recording documents
A WhatsApp channel – this now exists! Contact joe@ with your mobile number to be added to the group
A Slack channel for impact – perhaps with a weekly ‘hour of chat’ a la Bureau Local
A training course on impact – Just Enterprise’s training for non-profits was recommended and we could work with them to develop something specific to news providers;
A regular catchup among people interested in impact, perhaps as a way to nudge to keep impact records, but also an opportunity to share questions, ideas, experience and to hear from experts
Office hours / a coaching role - whereby the coach would regularly check-in with organisations seeking to track impact, to ask a set range of questions, the weekly answers to which would build up a record of impact
Some kind of combination of the previous two ideas
One Big Impact - A universal approach to measuring impact, where every publisher signs up to use a similar impact tracker, which all feed into an overarching ‘sector-wide tracker’ - for example, every publisher could use a similar reader survey, and then that data could be aggregated for benchmarking/comparison and enable claims for the sector as a whole
An exchange or platform for facilitating partnerships between news providers and universities, some of whom may have small, easy-access impact research budgets.
PINF might be able to take on some of these roles – and we’d also be keen to hear from others with an interest in these roles. Following the end of the series of five zoom calls, PINF will reflect on how best we can help boost these five projects/themes.
Links shared on the call
Clydesider – Social Impact Report
Flip Finance - Future News Impact & Evaluation Report
Social Streets – The Social Value of Local Journalism
Charitable Journalism Project - ‘News Deserts in the UK’
LION Publishers - How to measure and market your impact