Teaching

Why Many Academic Interventions Don’t Have Staying Power—and What to Do About It

By Sarah Schwartz — October 13, 2022 5 min read
Photo of teacher helping young student.
  • Save to favorites
  • Print

Some good things never last.

It’s a well-established research finding that the effects of many academic interventions, from extra math help for a few weeks to some years-long preschool programs, can “fade out” over time. The additional supports can raise student achievement in the short run, but after a few years, the kids who received it aren’t any better off than the kids who didn’t.

This pattern raises thorny questions for school and district leaders, especially now as school systems have poured COVID relief funding into efforts to accelerate student learning.

What kinds of interventions are most likely to show persistent effects? Are there ways to make sure that the short-term gains in academics or student well-being will last?

Now, scientists have proposed some answers. In a panel this week hosted by the Association for Psychological Science, two researchers discussed the available evidence.

The presenters—Drew Bailey, an associate professor at the University of California, Irvine’s education school, and David Yeager, an associate professor of psychology at the University of Texas at Austin—also wrote a 2020 paper on this topic.

Here are a few takeaways for educators from the paper and the panel.

Most interventions show some fade out, but not all. Measurement is one factor

One example of fade out comes from a study of a high-quality preschool math curriculum in 2011. At the end of preschool, students who had experienced this curriculum had stronger math skills than students who hadn’t. But by the end of 4th grade, those differences had almost completely disappeared.

In this case, that gap closed because students who never had the early math curriculum caught up: In the intervening years between preschool and 4th grade, their math achievement grew at a faster rate than that of their peers’ who did have it.

This is one reason why fade out occurs in educational interventions, the researchers write—other students who didn’t get the intervention eventually reap the same benefits through regular teaching. Another is that learning doesn’t transfer: A lot of what students do in school requires the interaction of different skills. If an intervention only targets one discrete skill, it may not lead to overall increases in learning.

The effects of some interventions persist, though—usually, ones that alter students’ lives in big ways, like adding extra years of school.

And some interventions fade out at first, but then see other effects emerge over time.

A well-known example of this last category is the Perry Preschool Program study, which provided a group of low-income children in Michigan with high-quality preschool and then tracked their outcomes over time. A boost in IQ for Perry students compared to their peers faded out, but later, researchers found that Perry students had an increase in lifetime earnings compared to non-Perry students.

The emergence of long-term effects highlights another point: There are often benefits to educational interventions that can’t be captured by test scores.

“A critical perspective that we often overlook when trying to evaluate the efficacy of a program or initiative is [that] we don’t step back and try to think about what the primary goal of the program is, and what it is designed to do,” Matthew Kraft, an associate professor of education and economics at Brown University, said in an interview with Education Week.

Test scores are a readily available data point, and as such are often used to evaluate outcomes. But that’s not always the best approach, Kraft said. The effectiveness of a mentoring program, for example, might be better measured by looking at student behavior, persistence, or engagement with the juvenile justice system.

“There are a whole lot of ways that schools have a meaningful impact on students’ lives,” Kraft said.

Interventions with persistent effects may share some key characteristics

Given this variation, how can education leaders know which interventions are likely to fade out, and which might have staying power? Bailey, Yeager, and their colleagues suggest that interventions with persistent effects target skills that meet three criteria:

  1. These skills are malleable, meaning it’s possible for interventions to help students improve on them in the first place;
  2. They’re fundamental—they serve as building blocks for other skills;
  3. And the skills wouldn’t have developed if not for the intervention.

Early reading intervention is one example cited in the paper. Foundational reading skills like phonemic awareness and phonics are malleable—students can improve with explicit instruction. They’re also fundamental, because being able to decode the words on the page is a prerequisite to all reading.

For many students, the regular teaching they get on these skills will be enough to master them. But for students who struggle, the intervention gives them an opportunity to develop these skills that they might not get in a whole class setting.

Other interventions can be effective, too, even if they don’t share these characteristics—as long as they’re deployed at the right time.

Bailey and colleagues write that there are some “gateways” in the education system that decide students’ long-term trajectories—whether they graduate high school, for instance, or enroll in college. Those are smart places to time interventions, they say.

For example, one pair of studies demonstrated that taking two periods of math courses instead of one had a big effect on 9th grade students’ educational attainment, but none for 6th graders who underwent the same intervention.

One possible reason for this, Bailey and his co-authors write, is that 9th grade algebra is a gateway course—whether students pass it or not can expand or limit their future high school options.

Cost-benefit analyses can support district decisionmaking

Just because an intervention has a big effect doesn’t mean it’s the right one for every circumstance, the researchers write.

“We should always consider those effects relative to the costs,” said Kraft, who authored a 2020 paper on interpreting the policy relevance of different effect sizes. “Sometimes programs that have very modest or even small impacts may actually be very good investments because they are very inexpensive.”

Another consideration for school districts is how easy an intervention is to sustain and scale. Sometimes it’s just hard to find people who can keep an amazing program with big effects going long-term, Kraft said.

Still, he said, cost-benefit analyses shouldn’t drive decisionmaking alone. Sometimes expensive shifts, like hiring more high-quality teachers, are necessary.

“I think we also have to take a values based approach,” Kraft said. “What do we as a society think we should be investing in, and how do we invest in it?”

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Budget & Finance Webinar
Innovative Funding Models: A Deep Dive into Public-Private Partnerships
Discover how innovative funding models drive educational projects forward. Join us for insights into effective PPP implementation.
Content provided by Follett Learning
Budget & Finance Webinar Staffing Schools After ESSER: What School and District Leaders Need to Know
Join our newsroom for insights on investing in critical student support positions as pandemic funds expire.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
How can districts build sustainable tutoring models before the money runs out?
District leaders, low on funds, must decide: broad support for all or deep interventions for few? Let's discuss maximizing tutoring resources.
Content provided by Varsity Tutors for Schools

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Opinion Better Classroom Management Can't Wait. How to Make Changes Now
The three “truths” about managing student behavior that every teacher should unlearn today.
Andrew Kwok
4 min read
A grid of classroom elements with lines flowing in and out of the segments.
iStock/Getty Images + Education Week
Teaching Opinion 4 Instructional Strategies Teachers Can Count On
Students can understand more challenging concepts when they have multiple opportunities to see how the content relates to other standards.
12 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Teaching Opinion 4 Ways to Use Student Curiosity to Deepen Learning
These intentional shifts in the classroom can help teachers foster engagement and inquiry, writes a learning specialist.
Ben Talsma
4 min read
Kid Characters Observe Sky with Moon, Milky Way and Reach for the stars!
iStock/Getty Images
Teaching Opinion 5 Simple Tips for Making an Outsized Impact on Students
There are a few things you can do at the start of a lesson to build student trust.
10 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty