Opinion Blog

Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Policy & Politics Opinion

What Policymakers and Practitioners Get Wrong About Education Research

By Rick Hess — April 07, 2022 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

Last week, drawing on a survey we conducted of the 2022 RHSU Edu-Scholars, I shared some advice they had to offer on how to be an effective researcher. They also had some interesting thoughts for practitioners and policymakers—especially on the question of what they tend to get wrong when applying education research. As someone who has spent many years thinking about the complex relationship between research, practice, and policy, I thought there were several points worth sharing.

A number of respondents noted that policymakers and practitioners frequently overstate the certainty of research findings while giving short shrift to context.

One scholar warned that policymakers and practitioners “fail to appreciate that research is rarely the final word on an issue; findings are situated in place and time,” another that it’s a mistake to assume that even “causal impact studies tell the correct or complete story,” and a third that we ought not regard “the latest high-profile study with the most extreme results as the last word.” On a similar note, another scholar wrote, “Using one or a few studies to make policy will almost inevitably go awry. Context matters, and it’s seldom wise to assume a finding will work as projected across all, or even most, contexts.”

A second major theme was that policymakers and practitioners are rarely equipped to understand the research they are given.

One scholar opined, “They neither understand mathematical reasoning nor statistical reasoning and as a result are easily duped. Even when exposed to empirical evidence, they are unable to reason.” Another argued, “Factors outside of schools account for most of the problems [policymakers] are trying to remedy. When pressed, I believe most education researchers will admit this, but since factors outside of schools are so hard to impact with policy, we mostly ignore them.”

And a third scholar mused, “It is very difficult for practitioners to access information on what works, under what conditions. Often, they hear more from companies and people trying to sell them something and hear a lot less from researchers. It’s less about them getting something wrong as it is a crowded marketplace with no one system for vetting different practices and policies.”

A third thread is the reality that school improvement is often incremental and exhausting, but that policymakers and practitioners have incentives to seek dramatic solutions.

One scholar argued that school improvement is harder than it may appear: “What appear like small effects are really quite impressive. Moving the needle in education is HARD.” Another noted that policymakers fall into the danger of “assuming schools are more powerful than they are and ignoring profound consequences of before-school and out-of-school influences. Often, the policymakers and superintendents make absurd promises, producing inevitable disappointment.”

A third scholar lamented that policymakers and practitioners tend to “focus too much on the bottom-line results, and, when things seem to fail to have effects, they focus too little on the reasons why.” In short, as another put it, “Policymakers let their hope for a relatively simple intervention that will overwhelm the complexities of the problem get the better of their common sense.”

Education research is a tool. It’s only as good as the hand that wields it. Lousy research is a poor tool, but even good research can be destructive if used ineptly. It’s important for researchers to appreciate this, and even more vital for educators and policymakers to do so.

Please note that answers were lightly edited for grammar and spelling.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Student Well-Being K-12 Essentials Forum Boosting Student and Staff Mental Health: What Schools Can Do
Join this free virtual event based on recent reporting on student and staff mental health challenges and how schools have responded.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
Practical Methods for Integrating Computer Science into Core Curriculum
Dive into insights on integrating computer science into core curricula with expert tips and practical strategies to empower students at every grade level.
Content provided by Learning.com

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Law & Courts Wisconsin Teachers Sue to Restore Collective Bargaining Rights
The lawsuit takes fresh aim at a 2011 law that severely restricted bargaining, and has survived several legal challenges since.
6 min read
Wisconsin Education Association Council (WEAC) vice president Betsy Kippers leads a chant during a rally to protest Governor Scott Walker's budget repair bill, at the Brown County Courthouse in downtown Green Bay on February 16, 2011.
Betsy Kippers, vice president of the Wisconsin Education Association Council, leads a chant during a rally to protest Gov. Scott Walker's budget repair bill, at the Brown County Courthouse in downtown Green Bay on February 16, 2011.
H. Marc Larson/The Green Bay Press-Gazette via AP
Law & Courts What Sandra Day O'Connor Did to Shape School Law and Civics Education
O'Connor wrote influential opinions on affirmative action, Title IX, and other education issues. Then she tirelessly worked on civics.
10 min read
Justice Sandra Day O'Connor listens as Justice Ruth Bader Ginsburg pays tribute to O'Connor's advocacy work on behalf of civic education, impact on female judges and justice for women and girls worldwide at the Seneca Women Global Leadership Forum at the National Museum of Women in the Arts, on April 15, 2015 in Washington.
Justice Sandra Day O'Connor listens to a tribute to her advocacy work on behalf of civics education and women's role in the legal profession at the National Museum of Women in the Arts, on April 15, 2015, in Washington.
Kevin Wolf/Invision for Seneca Women via AP Images
States DeSantis vs. Newsom: How K-12 Schools Fared in the 'Red vs. Blue State Debate'
The Florida and California governors sparred over book bans, school closures, and parental rights during their Fox News debate.
5 min read
Left: California Gov. Gavin Newsom speaks at a press conference in Beijing on Oct. 25, 2023. Right: Republican presidential candidate Ron DeSantis speaks at a Town Hall event at Tempesta's in Keene, N.H., on Nov. 21, 2023.
Left: California Gov. Gavin Newsom speaks at a press conference in Beijing on Oct. 25, 2023. Right: Republican presidential candidate Ron DeSantis speaks at a Town Hall event at Tempesta's in Keene, N.H., on Nov. 21, 2023.
Left: Ng Han Guan/AP; Right: Kristopher Radder/The Brattleboro Reformer via AP
International What the Research Says It's Not Just U.S. Students. Civics Scores Have Dropped Around the World
Eighth graders are less engaged and knowledgeable about government than they were before the pandemic, a global study finds.
5 min read
vote ballot initiatives money 1371378601 01
LAUDISENO/iStock/Getty and EdWeek