Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Teaching Opinion

The Promises and Limits of ‘Evidence-Based Practice’

Evidence should help guide practice, but it cannot—and should not—try to determine it
By Rick Hess — April 18, 2023 6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

In “Straight Talk with Rick and Jal,” Harvard University’s Jal Mehta and I examine some of the reforms and enthusiasms that permeate education. In a field full of buzzwords and jargon, our goal is simple: Tell the truth, in plain English, about what’s being proposed and what it might mean for students, teachers, and parents. We may be wrong and will frequently disagree, but we’ll try to be candid and ensure that you don’t need a Ph.D. in eduspeak to understand us.

Today’s topic is “evidence-based practice.”

Rick

Rick: There’s a lot of enthusiasm for “evidence-based practice.” I get it. I’d much rather schools employ evidence-based practices than “evidence-free” or “demonstrably ineffective” ones. But the question of just what an evidence-based practice is has loomed large as districts spend vast sums of pandemic aid on “evidence-based” interventions intended to combat learning loss or promote social and emotional well-being. After all, the evidence behind evidence-based practice can be surprisingly squishy. Even medical researchers, with their deep pockets and fancy lab equipment, change their minds with distressing regularity on things like exercise, alcohol consumption, or diet. The fact that a study concludes X doesn’t mean that another study won’t conclude Y a few years later.

In 2015, researchers attempted to replicate 97 studies with statistically significant results and found that more than a third couldn’t be duplicated. When someone took the original data and reran the study, the results disappeared. Moreover, even when something “works,” replication can be a huge challenge. Imagine that a vaccine is shown to “work” when patients got a 100-milligram dosage exactly 28 days apart. If that’s the deal, that’s what the evidence supports. If those exact instructions aren’t followed, the evidence becomes unreliable. Few educational interventions are understood or implemented that precisely; they’re usually more like public-health measures like masking or social distancing, where evidence tends to be more suggestive than definitive.

Now, I don’t mean to come off as “anti-evidence.” I think the research behind the science of reading is extensive, robust, and compelling, and it should be treated that way. Indeed, the thing I most appreciated about Russ Whitehurst’s tempestuous tenure as founding director at the Institute of Education Sciences was his fight to elevate rigorous evidence among researchers and practitioners. While Russ’ efforts infuriated many who like clothing their favored practices in the garb of evidence, I thought his efforts were overdue and important.

So, I guess I embrace the principle of evidence-based practice, but I’m not sure we always know what that even means. How about you?

Jal: I like the idea of evidence-informed practice. That centers the notion that there is some evidence that educators should take into consideration but also the reality that there is much about any given situation that is context-specific and depends on the expertise of the local practitioner.

Research on teaching suggests that teachers make at least 1,000 decisions a day, and research on “street-level bureaucrats”—teachers, cops, social workers—similarly finds that enacting a practice means making lots of discretionary decisions, often with insufficient resources and multiple conflicting imperatives. Given that, the idea that there is “evidence” that could guide each and every one of those decisions is just wildly unrealistic. There is also the fact that there are some pretty fundamental value disagreements in education that can’t be reconciled through empirical evidence.

One touchstone I find useful here is Dewey’s view that practitioners knowing the science of a particular area expands rather than contracts their range of action. People think that the equation is something like “science determines action” whereas it is more like “science helps you perceive more aspects of the situation, which enables better and more sophisticated contextual judgments.” For example, if you know that kids who can’t read frequently cover that up with compensatory strategies, you might be more discerning in who needs help with reading. This view centers the idea that teaching is complex professional work that can be informed by evidence, not dictated by it.

Rick: This is all really compelling. I especially like the “1,000 decisions” framing. I’ve always been struck that seasoned teachers talk about their successful handling of a dicey moment in ways that remind me of athletes discussing a big play: “It all happened quickly; I saw the opportunity and I reacted.” In both cases, accumulated experience is being instantly, fluidly applied. It’s all informed to some degree by research and training (or practice and film study), but the actions are instinctive.

Most of the day-to-day work of teaching is like that, I think. It’s like a canvas tarp stretched over more formal scaffolding of scope, sequence, and assessment. The evidence can be built into the scaffolding but usually only indirectly shapes the tarp. The “evidence-based” discussion can be so frustrating, as you note, because there’s good evidence about parts of that underlying structure (like how to build phonemic awareness) but much less that reliably speaks to the bulk of what teachers do each day.

In fact, much of what gets touted as “evidence-based” practice just isn’t. Rigorous research can and should eventually influence practice but only after a long, gradual accumulation of evidence compiled by many researchers employing various methods. That’s not the norm in education research. Indeed, when education advocates assert that a practice is “evidence-based,” the claims are frequently based on a handful of studies conducted by a small coterie of researchers. At other times, they cite research with seeming disregard for its validity. And, at least for me, the “What Works Clearinghouse” hasn’t alleviated these concerns; if anything, it seems to me that inconsistent standards have allowed dubious findings to be stamped as “evidence-based.”

Jal: Well, this very much gets to the policy conversation and when evidence is often used as a weapon or a cudgel rather than as a way of investigating what might work. The “science of reading” seems to have been boiled down to “phonics works” by conservatives who are angry, perhaps justifiably so, that progressives and ed. schools have been overly resistant to incorporating phonics into their instruction. But as a matter of research, there is a lot more to reading than phonics—including vocabulary, comprehension, fluency, and so forth.

And then, if we were to zoom out further, we want kids to read, yes, but we also want them to develop a love of reading, which would draw on a whole other body of literature (and practical knowledge) about intrinsic and extrinsic motivation. So, even in an area where more science might be helpful, there is still the question of which science we are centering.

Despite the obvious appeal of drawing on evidence (climate change, cough cough), I think evidentiary claims are too frequently used to close conversations rather than open them. In my experience, the best educators and leaders see lots of complexity, consider context, and artfully weave together different approaches to solve particular problems. Whereas people who are loudest about “evidence-based practices,” ironically tend to be more ideologues who have a few preferred solutions that they think can address every problem. Evidence should help to guide our work, but it cannot—and should not—try to determine it.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Educators Say Social Media Hurts Their Colleagues' Social Skills. Their Own? Not as Much
What educators might be missing when they say social media has a negative impact on their students and colleagues.
3 min read
Tight cropped photo of a young professional male using a smartphone which is illuminated with floating social media icons above.
iStock/Getty
Teaching Opinion An Interview With Educator and Author Jonathan Kozol
In his new book, longtime activist Jonathan Kozol dives further into school inequity and offers solutions.
6 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Teaching Opinion 4 Ways to Present STEM Role Models Girls Will Find Inspiring
Merely exposing students to female scientists isn’t enough to get them to pursue careers in STEM.
Lisa M.P. Munoz & Eva Pietri
4 min read
Images shows a stylized artistic landscape with soothing colors.
Getty
Teaching Opinion What Teachers Can Learn From Students
Group work can help students make friends and provide a foundation for building future collaborations and relationships.
9 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty