Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Policy & Politics Opinion

What Policymakers and Practitioners Get Wrong About Education Research

By Rick Hess — April 07, 2022 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

Last week, drawing on a survey we conducted of the 2022 RHSU Edu-Scholars, I shared some advice they had to offer on how to be an effective researcher. They also had some interesting thoughts for practitioners and policymakers—especially on the question of what they tend to get wrong when applying education research. As someone who has spent many years thinking about the complex relationship between research, practice, and policy, I thought there were several points worth sharing.

A number of respondents noted that policymakers and practitioners frequently overstate the certainty of research findings while giving short shrift to context.

One scholar warned that policymakers and practitioners “fail to appreciate that research is rarely the final word on an issue; findings are situated in place and time,” another that it’s a mistake to assume that even “causal impact studies tell the correct or complete story,” and a third that we ought not regard “the latest high-profile study with the most extreme results as the last word.” On a similar note, another scholar wrote, “Using one or a few studies to make policy will almost inevitably go awry. Context matters, and it’s seldom wise to assume a finding will work as projected across all, or even most, contexts.”

A second major theme was that policymakers and practitioners are rarely equipped to understand the research they are given.

One scholar opined, “They neither understand mathematical reasoning nor statistical reasoning and as a result are easily duped. Even when exposed to empirical evidence, they are unable to reason.” Another argued, “Factors outside of schools account for most of the problems [policymakers] are trying to remedy. When pressed, I believe most education researchers will admit this, but since factors outside of schools are so hard to impact with policy, we mostly ignore them.”

And a third scholar mused, “It is very difficult for practitioners to access information on what works, under what conditions. Often, they hear more from companies and people trying to sell them something and hear a lot less from researchers. It’s less about them getting something wrong as it is a crowded marketplace with no one system for vetting different practices and policies.”

A third thread is the reality that school improvement is often incremental and exhausting, but that policymakers and practitioners have incentives to seek dramatic solutions.

One scholar argued that school improvement is harder than it may appear: “What appear like small effects are really quite impressive. Moving the needle in education is HARD.” Another noted that policymakers fall into the danger of “assuming schools are more powerful than they are and ignoring profound consequences of before-school and out-of-school influences. Often, the policymakers and superintendents make absurd promises, producing inevitable disappointment.”

A third scholar lamented that policymakers and practitioners tend to “focus too much on the bottom-line results, and, when things seem to fail to have effects, they focus too little on the reasons why.” In short, as another put it, “Policymakers let their hope for a relatively simple intervention that will overwhelm the complexities of the problem get the better of their common sense.”

Education research is a tool. It’s only as good as the hand that wields it. Lousy research is a poor tool, but even good research can be destructive if used ineptly. It’s important for researchers to appreciate this, and even more vital for educators and policymakers to do so.

Please note that answers were lightly edited for grammar and spelling.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Data Webinar
Drive Instruction With Mastery-Based Assessment
Deliver the right data at the right time—in the right format—and empower better decisions.
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Profession Webinar
How Does Educator Well-Being Impact Social-Emotional Awareness in Schools?
Explore how adult well-being is key to promoting healthy social-emotional behaviors for students. Get strategies to reduce teacher stress.
Content provided by International Baccalaureate
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure Webinar
A New Era In Connected Learning: Security, Accessibility and Affordability for a Future-Ready Classroom
Learn about Windows 11 SE and Surface Laptop SE. Enable students to unlock learning and develop new skills.
Content provided by Microsoft Surface

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Law & Courts Conservatives’ Checklist: U.S. Supreme Court Education Decisions to Overrule
Here are five education issues that could be targets for reconsideration if Roe v. Wade falls.
3 min read
The Supreme Court in Washington, Dec. 3, 2021. The Supreme Court has turned away a plea from parents to block a new admissions policy at a prestigious high school in northern Virginia that a lower court had found discriminates against Asian American students.
The U.S. Supreme Court in Washington on Dec. 3, 2021.
J. Scott Applewhite/AP
Accountability Timeline: How Federal School Accountability Has Waxed and Waned
From its origins in the 1990s to the most-recent tack, see how the federal approach to accountability has shifted.
4 min read
President George W. Bush, left, participates in the swearing-in ceremony for the Secretary of Education Margaret Spellings, center, at the U.S. Dept. of Education on Jan. 31, 2005 in Washington. On the far right holding a bible is her husband Robert Spellings.
President George W. Bush, left, participates in the swearing-in ceremony for the Secretary of Education Margaret Spellings, center, at the U.S. Dept. of Education on Jan. 31, 2005 in Washington. On the far right holding a bible is her husband Robert Spellings.
AP Photo/Pablo Martinez Monsivais
Accountability School Accountability Is Restarting After a Two-Year Pause. Here's What That Means
For a moment, the COVID-19 pandemic succeeded in doing what periodic protests about school accountability couldn't: Halting it.
10 min read
Illustration of a gauge.
4zevar/iStock/Getty
Education Funding What America Spends on K-12: The Latest Federal Snapshot
About 93 percent of K-12 spending came from state and local sources in 2019-20—but more-recent year totals will reflect federal relief aid.
2 min read