The art, science and mystery of nonprofit news assessment

This grant-funded research on measuring the impact of nonprofit journalism won the Society for Professional Journalists’ Sigma Delta Chi award for excellence in research in 2013. / Investigative Reporting Workshop

by Charles Lewis and Hilary Niles

Abstract

This report seeks to answer the two-pronged question, “What is ‘impact,’ and how can it be measured consistently across nonprofit newsrooms?”

A review of recent, relevant literature and our informal conversations with experts in the field reveal growing ambitions toward the goal of developing a common framework for assessing journalism’s impact, yet few definitive conclusions about how exactly to reach that framework.

IRW-measuring-impact-graphic-2This is especially the case when journalism’s “impact” is defined by its ultimate social outcomes — not merely the familiar metrics of audience reach and website traffic.

As with all journalism, the frame defines the story, and audience is all-important. Defining “impact” as a social outcome proves a complicated proposition that generally evolves according to the constituency attempting to define it. Because various stakeholders have their own reasons for wanting to measure the impact of news, understanding those interests is an essential step in crafting measurement tools and interpreting the metrics they produce.

Limitations of impact assessment arise from several sources: the assumptions invariably made about the product and its outcome; the divergent and overlapping categories into which nonprofit journalism falls in the digital age; and the intractable problem of attempting to quantify “quality.”

These formidable challenges, though, don’t seem to deter people from posing and attempting to find answers to the impact question.

Various models for assessing impact are continually being tinkered with, and lessons from similar efforts in other fields offer useful insight for this journalistic endeavor. And past research has pointed to specific needs and suggestions for ways to advance the effort. From all of this collective wisdom, several principles emerge as the cornerstones upon which to build a common framework for impact assessment.

// Read the full report.

Related press

  • The Foundation Center // July 23, 2013 // Read More
  • “The mystery and measurement of nonprofit-news impacts” // by Josh Wilson, Investigative News Network // July 23, 2013 // Read More
  • “Journalists and their funders: Whose job is it to measure impact, and how should it be done?” // by Caroline O’Donovan, Nieman Journalism Lab // July 24, 2013 // Read More
  • “Why it’s time to stop romanticizing & begin measuring investigative journalism’s impact” // by Rick Edmonds, The Poynter Institute // August 19, 2013 // Read More
  • “Measuring the impact of journalism: 10 questions with Charles Lewis and Hilary Niles” // by Millie Tran, American Press Institute // August 19, 2013 // Read More
  • “Can Journalism’s Impact Be Measured?” // by Kevin Davis, NetNewsCheck // September 5, 2013 // Read More

Leave a Reply

Your email address will not be published. Required fields are marked *