Education Opinion

Evaluation in Context: Advice for Educational Non-Profits on Telling Stories with Data

By Justin Reich — January 18, 2014 3 min read
  • Save to favorites
  • Print

Educational non-profits who depend upon philanthropists--individuals or foundations--sell products to those philanthropists. Rather than selling services directly to those philanthropists, they sell stories about those services. The kid gets the new treatment, or resource, or program, or widget, or whatever, and the philanthropist buys a story from the non-profit about the kid and her experiences.

There is a sea-change happening in the stories that philanthropists are buying.

In the past, philanthropists bought anecdotes of change and transformation. “Here’s the life of kid X or teacher Y before experiencing program Z, and here’s how those lives have been enriched and trasformed.” That is still true to some extent, but increasingly, philanthropists want to purchase stories about data. They want to purchase stories of impact, scale, and growth that are told in numbers and spreadsheets, rather than stories of personal transformation and revelation told through words and faces.

Let’s set aside a moment the question of whether or not this change is for the best. If you are an educational non-profit, it doesn’t matter whether you like this change or not; the philanthropic marketplace is in the midst of a major shift. You need to be able to tell your story with data.

Along with three colleagues from the Berkman Center for Internet & Society, I’ve written a short brief, “Evaluation in Context,” for educational non-profits to help them think through strategies for program evaluation in the face of these changes. It provides a framework for thinking through the challenges of quantiative evaluation: from goal-setting, to imagining indicators of progress, to gathering data creatively while under all the rigors of running a small educational program.

The abstract of the paper is below and the full brief is freely available on SSRN. It was specifically written for grantees of the McCormick Foundation’s Why News Matters program, but I hope others beyond that circle find it useful.

Evaluation is the systematic investigation of the effects that your program has on the lives of people touched by your program. The process of designing an evaluation system benefits your program by clarifying your goals and identifying indicators of progress towards your goals. The results of evaluation provide program staff with important evidence that can shape a reflective process of iteration and improvement. The products of evaluation are also vital for demonstrating to philanthropists and grant-makers the value of their charitable investments.

For many non-profit programs, evaluation is foreign and scary. We plunge into public service out of a passion for serving people, not because we want to gin up data for analysis. For those whose work depends upon the charity of others, evaluations run the risk of revealing weaknesses and limitations. Many people can be intimidated by the quantitative aspects of evaluation or feel that numerical summaries of work can obscure the transformative impact of programs on individual lives.

But the best non-profit agencies are learning organizations, committed to a constant cycle of experiment, inquiry, reflection, and refinement. By studying our programs, testing our assumptions, and gathering evidence of our impact and shortcomings, we have the opportunity to do our work even better. We can use evaluation to engage a growing audience of philanthropists and grant-makers who want to invest in programs committed to gathering evidence of their effectiveness.

For regular updates, follow me on Twitter at @bjfr and for my publications, C.V., and online portfolio, visit EdTechResearcher.

The opinions expressed in EdTech Researcher are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.