Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Policy & Politics Opinion

What Policymakers and Practitioners Get Wrong About Education Research

By Rick Hess — April 07, 2022 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

Last week, drawing on a survey we conducted of the 2022 RHSU Edu-Scholars, I shared some advice they had to offer on how to be an effective researcher. They also had some interesting thoughts for practitioners and policymakers—especially on the question of what they tend to get wrong when applying education research. As someone who has spent many years thinking about the complex relationship between research, practice, and policy, I thought there were several points worth sharing.

A number of respondents noted that policymakers and practitioners frequently overstate the certainty of research findings while giving short shrift to context.

One scholar warned that policymakers and practitioners “fail to appreciate that research is rarely the final word on an issue; findings are situated in place and time,” another that it’s a mistake to assume that even “causal impact studies tell the correct or complete story,” and a third that we ought not regard “the latest high-profile study with the most extreme results as the last word.” On a similar note, another scholar wrote, “Using one or a few studies to make policy will almost inevitably go awry. Context matters, and it’s seldom wise to assume a finding will work as projected across all, or even most, contexts.”

A second major theme was that policymakers and practitioners are rarely equipped to understand the research they are given.

One scholar opined, “They neither understand mathematical reasoning nor statistical reasoning and as a result are easily duped. Even when exposed to empirical evidence, they are unable to reason.” Another argued, “Factors outside of schools account for most of the problems [policymakers] are trying to remedy. When pressed, I believe most education researchers will admit this, but since factors outside of schools are so hard to impact with policy, we mostly ignore them.”

And a third scholar mused, “It is very difficult for practitioners to access information on what works, under what conditions. Often, they hear more from companies and people trying to sell them something and hear a lot less from researchers. It’s less about them getting something wrong as it is a crowded marketplace with no one system for vetting different practices and policies.”

A third thread is the reality that school improvement is often incremental and exhausting, but that policymakers and practitioners have incentives to seek dramatic solutions.

One scholar argued that school improvement is harder than it may appear: “What appear like small effects are really quite impressive. Moving the needle in education is HARD.” Another noted that policymakers fall into the danger of “assuming schools are more powerful than they are and ignoring profound consequences of before-school and out-of-school influences. Often, the policymakers and superintendents make absurd promises, producing inevitable disappointment.”

A third scholar lamented that policymakers and practitioners tend to “focus too much on the bottom-line results, and, when things seem to fail to have effects, they focus too little on the reasons why.” In short, as another put it, “Policymakers let their hope for a relatively simple intervention that will overwhelm the complexities of the problem get the better of their common sense.”

Education research is a tool. It’s only as good as the hand that wields it. Lousy research is a poor tool, but even good research can be destructive if used ineptly. It’s important for researchers to appreciate this, and even more vital for educators and policymakers to do so.

Please note that answers were lightly edited for grammar and spelling.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

States Republican and Democratic Governors Both Are Touting This K-12 Priority
Workforce readiness and career and technical education were the most common education themes in governors' state of the state addresses.
6 min read
Heidi Griebel and Josie Wahl participate in carpentry class at Career and Technical Education Academy in Sioux Falls, S.D., on Jan. 7, 2019.
Heidi Griebel and Josie Wahl participate in carpentry class at Career and Technical Education Academy in Sioux Falls, S.D., on Jan. 7, 2019. CTE programs were a core theme of several governors' state addresses in 2024.
Loren Townsley/The Argus Leader via AP
States School Chaplain Bills Multiply, Stirring Debate on Faith-Based Counseling
Proponents say school chaplains could help address a mental health crisis. Opponents raise concerns about religious coercion.
6 min read
Image of a bible sitting on top of a school backpack.
Canva
Law & Courts When Blocking Social Media Critics, School Officials Have Protections, Supreme Court Says
The court said public officials' own pages may be "state action," but only when they are exercising government authority.
6 min read
An American flag waves in front of the Supreme Court building on Capitol Hill in Washington, on Nov. 2, 2020.
An American flag waves in front of the Supreme Court building on Capitol Hill in Washington, on Nov. 2, 2020.
Patrick Semansky/AP
Education Funding Biden's Budget Proposes Smaller Bump to Education Spending
The president requested increases to Title I and IDEA, and funding to expand preschool access in his 2025 budget proposal.
7 min read
President Joe Biden delivers remarks on lowering prices for American families during an event at the YMCA Allard Center on March 11, 2024, in Goffstown, N.H.
President Joe Biden delivers remarks on lowering prices for American families during an event at the YMCA Allard Center on March 11, 2024, in Goffstown, N.H. Biden's administration released its 2025 budget proposal, which includes a modest spending increase for the Education Department.
Evan Vucci/AP