Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Policy & Politics Opinion

What Policymakers and Practitioners Get Wrong About Education Research

By Rick Hess — April 07, 2022 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

Last week, drawing on a survey we conducted of the 2022 RHSU Edu-Scholars, I shared some advice they had to offer on how to be an effective researcher. They also had some interesting thoughts for practitioners and policymakers—especially on the question of what they tend to get wrong when applying education research. As someone who has spent many years thinking about the complex relationship between research, practice, and policy, I thought there were several points worth sharing.

A number of respondents noted that policymakers and practitioners frequently overstate the certainty of research findings while giving short shrift to context.

One scholar warned that policymakers and practitioners “fail to appreciate that research is rarely the final word on an issue; findings are situated in place and time,” another that it’s a mistake to assume that even “causal impact studies tell the correct or complete story,” and a third that we ought not regard “the latest high-profile study with the most extreme results as the last word.” On a similar note, another scholar wrote, “Using one or a few studies to make policy will almost inevitably go awry. Context matters, and it’s seldom wise to assume a finding will work as projected across all, or even most, contexts.”

A second major theme was that policymakers and practitioners are rarely equipped to understand the research they are given.

One scholar opined, “They neither understand mathematical reasoning nor statistical reasoning and as a result are easily duped. Even when exposed to empirical evidence, they are unable to reason.” Another argued, “Factors outside of schools account for most of the problems [policymakers] are trying to remedy. When pressed, I believe most education researchers will admit this, but since factors outside of schools are so hard to impact with policy, we mostly ignore them.”

And a third scholar mused, “It is very difficult for practitioners to access information on what works, under what conditions. Often, they hear more from companies and people trying to sell them something and hear a lot less from researchers. It’s less about them getting something wrong as it is a crowded marketplace with no one system for vetting different practices and policies.”

A third thread is the reality that school improvement is often incremental and exhausting, but that policymakers and practitioners have incentives to seek dramatic solutions.

One scholar argued that school improvement is harder than it may appear: “What appear like small effects are really quite impressive. Moving the needle in education is HARD.” Another noted that policymakers fall into the danger of “assuming schools are more powerful than they are and ignoring profound consequences of before-school and out-of-school influences. Often, the policymakers and superintendents make absurd promises, producing inevitable disappointment.”

A third scholar lamented that policymakers and practitioners tend to “focus too much on the bottom-line results, and, when things seem to fail to have effects, they focus too little on the reasons why.” In short, as another put it, “Policymakers let their hope for a relatively simple intervention that will overwhelm the complexities of the problem get the better of their common sense.”

Education research is a tool. It’s only as good as the hand that wields it. Lousy research is a poor tool, but even good research can be destructive if used ineptly. It’s important for researchers to appreciate this, and even more vital for educators and policymakers to do so.

Please note that answers were lightly edited for grammar and spelling.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Special Education Webinar Reading, Dyslexia, and Equity: Best Practices for Addressing a Threefold Challenge
Learn about proven strategies for instruction and intervention that support students with dyslexia.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Personalized Learning Webinar
No Time to Waste: Individualized Instruction Will Drive Change
Targeted support and intervention can boost student achievement. Join us to explore tutoring’s role in accelerating the turnaround. 
Content provided by Varsity Tutors for Schools
Student Well-Being K-12 Essentials Forum Social-Emotional Learning: Making It Meaningful
Join us for this event with educators and experts on the damage the pandemic did to academic and social and emotional well-being.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal How a Divided Congress Will Influence K-12 Education Policy
The U.S. Senate and House of Representatives education committees will have new leaders in January.
6 min read
Sen. Bernie Sanders, I-Vt., speaks Monday, June 13, 2022, during a debate with Sen. Lindsey Graham, R-S.C, Hosted by Fox News at the The Edward M. Kennedy Institute in Boston for a debate intended to prove that bipartisanship isn't dead.
Sen. Bernie Sanders, I-Vt., speaks during a June debate with Sen. Lindsey Graham, R-S.C, at The Edward M. Kennedy Institute in Boston. Sanders is poised to become the chair of the Senate Health, Education, Labor, and Pensions Committee.
Josh Reynolds/AP
School Choice & Charters Charter School Enrollment Holds Steady After Big Early Pandemic Growth
The numbers show that most students who left their district schools in the first year of the pandemic did not return.
2 min read
Image of an empty classroom.
urfinguss/iStock/Getty
Education Funding What the Research Says Districts Are Spending More Per Student. Here's How to Make Sure All of Them Benefit
New studies suggest ways education leaders can make budgets bigger and more equitable.
4 min read
Educators delivering money.
iStock/Getty Images Plus
Law & Courts Supreme Court Won't Take Up Cases Seen as Expanding Schools' Liability for Sexual Harassment
The court rejected appeals from a school district and university about when educational institutions may be sued for sexual harassment.
4 min read
Thunder storm sky over the United States Supreme Court building in Washington DC.
iStock/Getty Images Plus