Education Data for Improvement, Not Accountability: Q&A With Paul LeMahieu

By Benjamin Herold — June 22, 2018 3 min read

Paul LeMahieu is a co-author of Learning to Improve: How America’s Schools Can Get Better at Getting Better. His work on bringing a “continuous improvement” approach to K-12 education spans a career that has included stints as Hawaii’s superintendent of education and the director of research at the National Writing Project. LeMahieu is currently the senior vice president of the Carnegie Foundation for the Advancement of Teaching, an early and ongoing leader in the continuous-improvement push.

In February, LeMahieu talked with Education Week about using educational data to support continuous improvement. The following Q&A has been edited for length and clarity.

How is research for continuous improvement different from that more traditional type of research?

Traditional research tells us something can work. It mostly aspires to build theory and to warrant claims of causation. But the very things you do to warrant that claim also limits and restricts the transfer and utility and generalizability of that knowledge.

Paul LeMahieu

Improvement science tells us how to make it work, over and over again, across contexts.

Our work basically is comprised of wedding together two other big sets of ideas: improvement science, and the power of networks.

Can you explain those two ideas?

Improvement science is not new. It’s been around for close to 100 years, but most of that time in business and industry. Over the last 25 or so years, we’ve seen improvement science in professions such as health care, child welfare services, criminal justice, other contexts that are more like education.

We think what we refer to as networked improvement communities are a uniquely effective social organization in which to do improvement work. They are rich sources of ideas and innovation. They also provide diverse contexts. And networks allow you to see patterns. We don’t lack for good ideas in education. We lack for methodology that rigorously helps us to implement those good ideas, so they succeed across contexts.

Most of the data infrastructure in K-12 today has been built for accountability purposes, not continuous improvement. What can be retrofitted, and what needs to be built new?

In an accountability context, the questions are, ‘Did you do what you were supposed to do, and did you realize the outcomes you hoped for?”

That’s not good enough for improvement.

The essential questions of improvement science are, “what works, for whom, under what condition?” To answer those, we need new forms of data. We need data that illuminate the conditions where an outcome is observed. That includes things such as what sort of policies exist in this environment, what sort of programs and practices are prevalent in this environment. We also need to look at so-called positive deviants, or rich places where positive things can be learned.

You’ve talked and written about “balancing measures” and “leading indicators.” What are they?

For improvement, you need both leading and lagging indicators. Take for example improving the teacher workforce. We often cares deeply about outcomes that can be quite distant, like teacher retention. But if our aim to help those teachers get better more quickly and to hold on to them, we don’t just need that lagging indicator. We also need a leading indicator, such as how many of them are staying on track. We need to build structures that capture these sort of things routinely.

How much work will that be?

We’re pretty far off from it. The insight is only now dawning on us.

How should we gauge the capacity of new digital data systems to help?

Follow the data. Ask, “Who is the system for, who gets the data, how are they able to use [them], and to what end?”

Do you expect to see much change in the next five years?

I’m actually quite optimistic. The movement now has some momentum. It’s starting to show up in places where programs and practices get shaped. ESSA makes provision for continuous improvement. The investment by the Gates Foundation is absolutely significant. I’m concerned about quality—what we don’t want is for this to be is this decade’s fad. But I think we’re ready to turn a corner with the dawning of improvement thinking.

Coverage of continuous-improvement strategies in education is supported in part by a grant from the Bill & Melinda Gates Foundation. Education Week retains sole editorial control over the content of this coverage.
Coverage of continuous-improvement strategies in education is supported in part by a grant from the Bill & Melinda Gates Foundation, at Education Week retains sole editorial control over the content of this coverage.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Here to Stay – Pandemic Lessons for EdTech in Future Development
What technology is needed in a post pandemic district? Learn how changes in education will impact development of new technologies.
Content provided by AWS
School & District Management Live Online Discussion A Seat at the Table: Strategies & Tips for Complex Decision-Making
Schools are working through the most disruptive period in the history of modern education, facing a pandemic, economic problems, social justice issues, and rapid technological change all at once. But even after the pandemic ends,

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Biden Education Team Squashes States' Push to Nix All Tests but Approves Other Flexibility
The department has telegraphed its decision to deny states' requests to cancel federally mandated tests for weeks.
3 min read
A first-grader learns keyboarding skills at Bayview Elementary School in San Pablo, Calif on March 12, 2015. Schools around the country are teaching students as young as 6 years old, basic typing and other keyboarding skills. The Common Core education standards adopted by a majority of states call for students to be able to use technology to research, write and give oral presentations, but the imperative for educators arrived with the introduction of standardized tests that are taken on computers instead of with paper and pencils.
The U.S. Department of Education denied some states' requests to cancel standardized tests this year. Others are seeking flexibility from some testing requirements, rather than skipping the assessments altogether.
Eric Risberg/AP
Accountability Explainer Will There Be Standardized Tests This Year? 8 Questions Answered
Educators want to know: Will the exams happen? If so, what will they look like, and how will the results be used?
12 min read
Students testing.
Accountability Opinion What Should School Accountability Look Like in a Time of COVID-19?
Remote learning is not like in person, and after nine months of it, data are revealing how harmful COVID-19 has been to children's learning.
6 min read
Image shows a speech bubble divided into 4 overlapping, connecting parts.
DigitalVision Vectors/Getty and Laura Baker/Education Week
Accountability State Schools Chiefs Push Biden for Wiggle Room on Accountability During Pandemic
State schools chiefs say it's necessary to change how they use scores from mandated annual tests during the unprecedented disruption created by the coronavirus pandemic.
4 min read
Image of students taking a test.