Teaching

Why Many Academic Interventions Don’t Have Staying Power—and What to Do About It

By Sarah Schwartz — October 13, 2022 5 min read
Photo of teacher helping young student.
  • Save to favorites
  • Print

Some good things never last.

It’s a well-established research finding that the effects of many academic interventions, from extra math help for a few weeks to some years-long preschool programs, can “fade out” over time. The additional supports can raise student achievement in the short run, but after a few years, the kids who received it aren’t any better off than the kids who didn’t.

This pattern raises thorny questions for school and district leaders, especially now as school systems have poured COVID relief funding into efforts to accelerate student learning.

What kinds of interventions are most likely to show persistent effects? Are there ways to make sure that the short-term gains in academics or student well-being will last?

Now, scientists have proposed some answers. In a panel this week hosted by the Association for Psychological Science, two researchers discussed the available evidence.

The presenters—Drew Bailey, an associate professor at the University of California, Irvine’s education school, and David Yeager, an associate professor of psychology at the University of Texas at Austin—also wrote a 2020 paper on this topic.

Here are a few takeaways for educators from the paper and the panel.

Most interventions show some fade out, but not all. Measurement is one factor

One example of fade out comes from a study of a high-quality preschool math curriculum in 2011. At the end of preschool, students who had experienced this curriculum had stronger math skills than students who hadn’t. But by the end of 4th grade, those differences had almost completely disappeared.

In this case, that gap closed because students who never had the early math curriculum caught up: In the intervening years between preschool and 4th grade, their math achievement grew at a faster rate than that of their peers’ who did have it.

This is one reason why fade out occurs in educational interventions, the researchers write—other students who didn’t get the intervention eventually reap the same benefits through regular teaching. Another is that learning doesn’t transfer: A lot of what students do in school requires the interaction of different skills. If an intervention only targets one discrete skill, it may not lead to overall increases in learning.

The effects of some interventions persist, though—usually, ones that alter students’ lives in big ways, like adding extra years of school.

And some interventions fade out at first, but then see other effects emerge over time.

A well-known example of this last category is the Perry Preschool Program study, which provided a group of low-income children in Michigan with high-quality preschool and then tracked their outcomes over time. A boost in IQ for Perry students compared to their peers faded out, but later, researchers found that Perry students had an increase in lifetime earnings compared to non-Perry students.

The emergence of long-term effects highlights another point: There are often benefits to educational interventions that can’t be captured by test scores.

“A critical perspective that we often overlook when trying to evaluate the efficacy of a program or initiative is [that] we don’t step back and try to think about what the primary goal of the program is, and what it is designed to do,” Matthew Kraft, an associate professor of education and economics at Brown University, said in an interview with Education Week.

Test scores are a readily available data point, and as such are often used to evaluate outcomes. But that’s not always the best approach, Kraft said. The effectiveness of a mentoring program, for example, might be better measured by looking at student behavior, persistence, or engagement with the juvenile justice system.

“There are a whole lot of ways that schools have a meaningful impact on students’ lives,” Kraft said.

Interventions with persistent effects may share some key characteristics

Given this variation, how can education leaders know which interventions are likely to fade out, and which might have staying power? Bailey, Yeager, and their colleagues suggest that interventions with persistent effects target skills that meet three criteria:

  1. These skills are malleable, meaning it’s possible for interventions to help students improve on them in the first place;
  2. They’re fundamental—they serve as building blocks for other skills;
  3. And the skills wouldn’t have developed if not for the intervention.

Early reading intervention is one example cited in the paper. Foundational reading skills like phonemic awareness and phonics are malleable—students can improve with explicit instruction. They’re also fundamental, because being able to decode the words on the page is a prerequisite to all reading.

For many students, the regular teaching they get on these skills will be enough to master them. But for students who struggle, the intervention gives them an opportunity to develop these skills that they might not get in a whole class setting.

Other interventions can be effective, too, even if they don’t share these characteristics—as long as they’re deployed at the right time.

Bailey and colleagues write that there are some “gateways” in the education system that decide students’ long-term trajectories—whether they graduate high school, for instance, or enroll in college. Those are smart places to time interventions, they say.

For example, one pair of studies demonstrated that taking two periods of math courses instead of one had a big effect on 9th grade students’ educational attainment, but none for 6th graders who underwent the same intervention.

One possible reason for this, Bailey and his co-authors write, is that 9th grade algebra is a gateway course—whether students pass it or not can expand or limit their future high school options.

Cost-benefit analyses can support district decisionmaking

Just because an intervention has a big effect doesn’t mean it’s the right one for every circumstance, the researchers write.

“We should always consider those effects relative to the costs,” said Kraft, who authored a 2020 paper on interpreting the policy relevance of different effect sizes. “Sometimes programs that have very modest or even small impacts may actually be very good investments because they are very inexpensive.”

Another consideration for school districts is how easy an intervention is to sustain and scale. Sometimes it’s just hard to find people who can keep an amazing program with big effects going long-term, Kraft said.

Still, he said, cost-benefit analyses shouldn’t drive decisionmaking alone. Sometimes expensive shifts, like hiring more high-quality teachers, are necessary.

“I think we also have to take a values based approach,” Kraft said. “What do we as a society think we should be investing in, and how do we invest in it?”

Events

Special Education Webinar Reading, Dyslexia, and Equity: Best Practices for Addressing a Threefold Challenge
Learn about proven strategies for instruction and intervention that support students with dyslexia.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Families & the Community Webinar
How Whole-Child Student Data Can Strengthen Family Connections
Learn how district leaders can use these actionable strategies to increase family engagement in their student’s education and boost their academic achievement.
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
College & Workforce Readiness Webinar
The School to Workforce Gap: How Are Schools Setting Students Up For Life & Lifestyle Success?
Hear from education and business leaders on how schools are preparing students for their leap into the workforce.
Content provided by Find Your Grind

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Knock Out Those Letters of Recommendation: Pro Tips From Teachers
Teachers dish on best practices for students' letters of recommendation.
3 min read
Image shows University Application Acceptance Notification Letter with ACCEPTED Stamp
E+/Getty
Teaching Opinion Sage Advice From Veteran Teachers to Those New to the Classroom
Teachers can learn a lot by tapping into their fellow teachers and other school staff as well as students.
15 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Teaching Opinion I'm a Student. Here's Why Group Work Feels So Unfair
I get a sinking feeling of dread from the fear of an uneven distribution of work. Here's what I wish teachers would do instead.
Yun F. Zheng
3 min read
Illustration of students meeting.
<b>F. Sheehan / Education Week / Getty</b>
Teaching How Bringing STEM Experts Into Your Classroom Can Help Engage Students
Inviting guest speakers to talk to students can provide higher levels of relevance to classroom instruction.
4 min read
Two Female College Students Building Machine In Science Robotics Or Engineering Class
iStock/Getty