Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Teaching Opinion

The Promises and Limits of ‘Evidence-Based Practice’

Evidence should help guide practice, but it cannot—and should not—try to determine it
By Rick Hess — April 18, 2023 6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

In “Straight Talk with Rick and Jal,” Harvard University’s Jal Mehta and I examine some of the reforms and enthusiasms that permeate education. In a field full of buzzwords and jargon, our goal is simple: Tell the truth, in plain English, about what’s being proposed and what it might mean for students, teachers, and parents. We may be wrong and will frequently disagree, but we’ll try to be candid and ensure that you don’t need a Ph.D. in eduspeak to understand us.

Today’s topic is “evidence-based practice.”

Rick

Rick: There’s a lot of enthusiasm for “evidence-based practice.” I get it. I’d much rather schools employ evidence-based practices than “evidence-free” or “demonstrably ineffective” ones. But the question of just what an evidence-based practice is has loomed large as districts spend vast sums of pandemic aid on “evidence-based” interventions intended to combat learning loss or promote social and emotional well-being. After all, the evidence behind evidence-based practice can be surprisingly squishy. Even medical researchers, with their deep pockets and fancy lab equipment, change their minds with distressing regularity on things like exercise, alcohol consumption, or diet. The fact that a study concludes X doesn’t mean that another study won’t conclude Y a few years later.

In 2015, researchers attempted to replicate 97 studies with statistically significant results and found that more than a third couldn’t be duplicated. When someone took the original data and reran the study, the results disappeared. Moreover, even when something “works,” replication can be a huge challenge. Imagine that a vaccine is shown to “work” when patients got a 100-milligram dosage exactly 28 days apart. If that’s the deal, that’s what the evidence supports. If those exact instructions aren’t followed, the evidence becomes unreliable. Few educational interventions are understood or implemented that precisely; they’re usually more like public-health measures like masking or social distancing, where evidence tends to be more suggestive than definitive.

Now, I don’t mean to come off as “anti-evidence.” I think the research behind the science of reading is extensive, robust, and compelling, and it should be treated that way. Indeed, the thing I most appreciated about Russ Whitehurst’s tempestuous tenure as founding director at the Institute of Education Sciences was his fight to elevate rigorous evidence among researchers and practitioners. While Russ’ efforts infuriated many who like clothing their favored practices in the garb of evidence, I thought his efforts were overdue and important.

So, I guess I embrace the principle of evidence-based practice, but I’m not sure we always know what that even means. How about you?

Jal: I like the idea of evidence-informed practice. That centers the notion that there is some evidence that educators should take into consideration but also the reality that there is much about any given situation that is context-specific and depends on the expertise of the local practitioner.

Research on teaching suggests that teachers make at least 1,000 decisions a day, and research on “street-level bureaucrats”—teachers, cops, social workers—similarly finds that enacting a practice means making lots of discretionary decisions, often with insufficient resources and multiple conflicting imperatives. Given that, the idea that there is “evidence” that could guide each and every one of those decisions is just wildly unrealistic. There is also the fact that there are some pretty fundamental value disagreements in education that can’t be reconciled through empirical evidence.

One touchstone I find useful here is Dewey’s view that practitioners knowing the science of a particular area expands rather than contracts their range of action. People think that the equation is something like “science determines action” whereas it is more like “science helps you perceive more aspects of the situation, which enables better and more sophisticated contextual judgments.” For example, if you know that kids who can’t read frequently cover that up with compensatory strategies, you might be more discerning in who needs help with reading. This view centers the idea that teaching is complex professional work that can be informed by evidence, not dictated by it.

Rick: This is all really compelling. I especially like the “1,000 decisions” framing. I’ve always been struck that seasoned teachers talk about their successful handling of a dicey moment in ways that remind me of athletes discussing a big play: “It all happened quickly; I saw the opportunity and I reacted.” In both cases, accumulated experience is being instantly, fluidly applied. It’s all informed to some degree by research and training (or practice and film study), but the actions are instinctive.

Most of the day-to-day work of teaching is like that, I think. It’s like a canvas tarp stretched over more formal scaffolding of scope, sequence, and assessment. The evidence can be built into the scaffolding but usually only indirectly shapes the tarp. The “evidence-based” discussion can be so frustrating, as you note, because there’s good evidence about parts of that underlying structure (like how to build phonemic awareness) but much less that reliably speaks to the bulk of what teachers do each day.

In fact, much of what gets touted as “evidence-based” practice just isn’t. Rigorous research can and should eventually influence practice but only after a long, gradual accumulation of evidence compiled by many researchers employing various methods. That’s not the norm in education research. Indeed, when education advocates assert that a practice is “evidence-based,” the claims are frequently based on a handful of studies conducted by a small coterie of researchers. At other times, they cite research with seeming disregard for its validity. And, at least for me, the “What Works Clearinghouse” hasn’t alleviated these concerns; if anything, it seems to me that inconsistent standards have allowed dubious findings to be stamped as “evidence-based.”

Jal: Well, this very much gets to the policy conversation and when evidence is often used as a weapon or a cudgel rather than as a way of investigating what might work. The “science of reading” seems to have been boiled down to “phonics works” by conservatives who are angry, perhaps justifiably so, that progressives and ed. schools have been overly resistant to incorporating phonics into their instruction. But as a matter of research, there is a lot more to reading than phonics—including vocabulary, comprehension, fluency, and so forth.

And then, if we were to zoom out further, we want kids to read, yes, but we also want them to develop a love of reading, which would draw on a whole other body of literature (and practical knowledge) about intrinsic and extrinsic motivation. So, even in an area where more science might be helpful, there is still the question of which science we are centering.

Despite the obvious appeal of drawing on evidence (climate change, cough cough), I think evidentiary claims are too frequently used to close conversations rather than open them. In my experience, the best educators and leaders see lots of complexity, consider context, and artfully weave together different approaches to solve particular problems. Whereas people who are loudest about “evidence-based practices,” ironically tend to be more ideologues who have a few preferred solutions that they think can address every problem. Evidence should help to guide our work, but it cannot—and should not—try to determine it.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Inside PLCs: Proven Strategies from K-12 Leaders
Join an expert panel to explore strategies for building collaborative PLCs, overcoming common challenges, and using data effectively.
Content provided by Otus
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Making Science Stick: The Engaging Power of Hands-On Learning
How can you make science class the highlight of your students’ day while
achieving learning outcomes? Find out in this session.
Content provided by LEGO Education
Teaching Profession Key Insights to Elevate and Inspire Today’s Teachers
Join this free half day virtual event to energize your teaching and cultivate a positive learning experience for students.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Download Teachers, Here's How to Build Stronger Relationships With Boys (Downloadable)
Boys are relational learners, experts say. Here are eight key strategies for how to reach them.
Jessica Arrow, a play-based learning kindergarten teacher, leads her kindergarten class back into their classroom from forest play time at Symonds Elementary School in Keene, N.H. on Nov. 7, 2024.
Jessica Arrow, a kindergarten teacher at Symonds Elementary School in Keene, N.H., leads her students back into their classroom from forest play time on Nov. 7, 2024. Boys crave strong relationships with their teachers, experts say.
Sophie Park for Education Week
Teaching Opinion 10 Actions Teachers Can Take Now That Trump Is President
On Day 1 of his second term, Trump issued orders that could negatively affect students. Here’s how to support them.
4 min read
Conceptual illustration of classroom conversations and fragmented education elements coming together to form a cohesive picture of a book of classroom knowledge.
Sonia Pulido for Education Week
Teaching Spotlight Spotlight on PreK-12 Problem-Based Learning
This Spotlight will help you learn how to prepare students for the workforce, partner with students on sustainability initiatives, and more.
Teaching Opinion How Teachers Can Care for Their Students and Themselves This Year (Downloadable)
A veteran teacher suggests 8 essential practices to benefit everyone in the classroom.
Justin Parmenter
1 min read
Education and Learning icons in the classroom or online. Idea knowledge of innovative technology, science, and mathematics.
iStock/Getty Images + Education Week