Why Many Academic Interventions Don’t Have Staying Power—and What to Do About It

By Sarah Schwartz — October 13, 2022 5 min read
Photo of teacher helping young student.
  • Save to favorites
  • Print

Some good things never last.

It’s a well-established research finding that the effects of many academic interventions, from extra math help for a few weeks to some years-long preschool programs, can “fade out” over time. The additional supports can raise student achievement in the short run, but after a few years, the kids who received it aren’t any better off than the kids who didn’t.

This pattern raises thorny questions for school and district leaders, especially now as school systems have poured COVID relief funding into efforts to accelerate student learning.

What kinds of interventions are most likely to show persistent effects? Are there ways to make sure that the short-term gains in academics or student well-being will last?

Now, scientists have proposed some answers. In a panel this week hosted by the Association for Psychological Science, two researchers discussed the available evidence.

The presenters—Drew Bailey, an associate professor at the University of California, Irvine’s education school, and David Yeager, an associate professor of psychology at the University of Texas at Austin—also wrote a 2020 paper on this topic.

Here are a few takeaways for educators from the paper and the panel.

Most interventions show some fade out, but not all. Measurement is one factor

One example of fade out comes from a study of a high-quality preschool math curriculum in 2011. At the end of preschool, students who had experienced this curriculum had stronger math skills than students who hadn’t. But by the end of 4th grade, those differences had almost completely disappeared.

In this case, that gap closed because students who never had the early math curriculum caught up: In the intervening years between preschool and 4th grade, their math achievement grew at a faster rate than that of their peers’ who did have it.

This is one reason why fade out occurs in educational interventions, the researchers write—other students who didn’t get the intervention eventually reap the same benefits through regular teaching. Another is that learning doesn’t transfer: A lot of what students do in school requires the interaction of different skills. If an intervention only targets one discrete skill, it may not lead to overall increases in learning.

The effects of some interventions persist, though—usually, ones that alter students’ lives in big ways, like adding extra years of school.

And some interventions fade out at first, but then see other effects emerge over time.

A well-known example of this last category is the Perry Preschool Program study, which provided a group of low-income children in Michigan with high-quality preschool and then tracked their outcomes over time. A boost in IQ for Perry students compared to their peers faded out, but later, researchers found that Perry students had an increase in lifetime earnings compared to non-Perry students.

The emergence of long-term effects highlights another point: There are often benefits to educational interventions that can’t be captured by test scores.

“A critical perspective that we often overlook when trying to evaluate the efficacy of a program or initiative is [that] we don’t step back and try to think about what the primary goal of the program is, and what it is designed to do,” Matthew Kraft, an associate professor of education and economics at Brown University, said in an interview with Education Week.

Test scores are a readily available data point, and as such are often used to evaluate outcomes. But that’s not always the best approach, Kraft said. The effectiveness of a mentoring program, for example, might be better measured by looking at student behavior, persistence, or engagement with the juvenile justice system.

“There are a whole lot of ways that schools have a meaningful impact on students’ lives,” Kraft said.

Interventions with persistent effects may share some key characteristics

Given this variation, how can education leaders know which interventions are likely to fade out, and which might have staying power? Bailey, Yeager, and their colleagues suggest that interventions with persistent effects target skills that meet three criteria:

  1. These skills are malleable, meaning it’s possible for interventions to help students improve on them in the first place;
  2. They’re fundamental—they serve as building blocks for other skills;
  3. And the skills wouldn’t have developed if not for the intervention.

Early reading intervention is one example cited in the paper. Foundational reading skills like phonemic awareness and phonics are malleable—students can improve with explicit instruction. They’re also fundamental, because being able to decode the words on the page is a prerequisite to all reading.

For many students, the regular teaching they get on these skills will be enough to master them. But for students who struggle, the intervention gives them an opportunity to develop these skills that they might not get in a whole class setting.

Other interventions can be effective, too, even if they don’t share these characteristics—as long as they’re deployed at the right time.

Bailey and colleagues write that there are some “gateways” in the education system that decide students’ long-term trajectories—whether they graduate high school, for instance, or enroll in college. Those are smart places to time interventions, they say.

For example, one pair of studies demonstrated that taking two periods of math courses instead of one had a big effect on 9th grade students’ educational attainment, but none for 6th graders who underwent the same intervention.

One possible reason for this, Bailey and his co-authors write, is that 9th grade algebra is a gateway course—whether students pass it or not can expand or limit their future high school options.

Cost-benefit analyses can support district decisionmaking

Just because an intervention has a big effect doesn’t mean it’s the right one for every circumstance, the researchers write.

“We should always consider those effects relative to the costs,” said Kraft, who authored a 2020 paper on interpreting the policy relevance of different effect sizes. “Sometimes programs that have very modest or even small impacts may actually be very good investments because they are very inexpensive.”

Another consideration for school districts is how easy an intervention is to sustain and scale. Sometimes it’s just hard to find people who can keep an amazing program with big effects going long-term, Kraft said.

Still, he said, cost-benefit analyses shouldn’t drive decisionmaking alone. Sometimes expensive shifts, like hiring more high-quality teachers, are necessary.

“I think we also have to take a values based approach,” Kraft said. “What do we as a society think we should be investing in, and how do we invest in it?”


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Pave the Path to Excellence in Math
Empower your students' math journey with Sue O'Connell, author of “Math in Practice” and “Navigating Numeracy.”
Content provided by hand2mind
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Recruitment & Retention Webinar
Combatting Teacher Shortages: Strategies for Classroom Balance and Learning Success
Learn from leaders in education as they share insights and strategies to support teachers and students.
Content provided by DreamBox Learning
Classroom Technology K-12 Essentials Forum Reading Instruction and AI: New Strategies for the Big Education Challenges of Our Time
Join the conversation as experts in the field explore these instructional pain points and offer game-changing guidance for K-12 leaders and educators.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Opinion Making Culturally Responsive Teaching Work: Zaretta Hammond Corrects 3 Big Misconceptions
The author and coach explains how to avoid several common mistakes in culturally responsive teaching.
10 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Teaching Learning 'Acceleration' Is Hard to Do. These Districts Are Tackling the Challenge
Despite the buzz, acceleration hasn’t proved to be as popular as other learning-recovery strategies.
9 min read
EL teacher Katina Tibbetts looks at her computer during a 5th grade level Wit and Wisdom lesson at East Veterans Memorial Elementary in Gloucester, Mass., on Sept. 20, 2023.
Katina Tibbetts, a teacher who specializes in instruction for English learners, works with 5th-grade students during a lesson at East Veterans Memorial Elementary in Gloucester, Mass., on Sept. 20, 2023.
Libby O'Neill for Education Week
Teaching Spotlight Spotlight on Student Engagement & Motivation
This Spotlight will help you with insights into educators’ strategies for engagement, explore how podcasts are boosting engagement, and more.