Student Achievement

What Two New Studies Reveal About Learning Recovery

By Sarah Schwartz — August 10, 2023 5 min read
Third-grade teaching assistant Keione Vance leads a reading session with a small group of students at Boyd Elementary School in Atlanta on Dec. 15, 2022. To address learning loss caused by the pandemic, Atlanta has been one of the only cities in the country to add class time – 30 minutes a day for three years.
  • Save to favorites
  • Print

Two new studies shed light on the promise—and challenges—of learning recovery strategies that schools adopted in the wake of the pandemic.

The key theme from the research is as much a truism as it is a prescription: Getting the implementation right really matters. And it’s also difficult.

One working paper, from researchers at the American Institutes for Research, Harvard University, and the assessment provider NWEA, analyzed eight districts’ summer 2022 learning programs. Researchers found that the programs had a small but positive effect on student math scores, and no effect on reading.

Another working paper, from researchers at Vanderbilt University and Stanford University, examined tutoring programs—specifically, what conditions led to successful implementation. They identified some commonalities: Programs that functioned well had the support of school leaders and a structured process for recruiting tutors and managing logistics.

Together, the two studies underscore that academic recovery strategies require sustained, coordinated effort, said Susanna Loeb, a professor in the Graduate School of Education at Stanford, and one of the authors on the tutoring study.

“Anything that you’re going to do in schools, whether it is high-impact tutoring or a summer program, in order to have the kinds of impact that we need to get students engaged in school and learning at a faster pace, … you’re going to need a lot of effort into implementation,” Loeb said.

Giving students many and varied opportunities for extra learning is important, too, said Emily Morton, a researcher at AIR’s National Center for Analysis of Longitudinal Data in Education Research and one of the authors on the summer learning study.

The summer programs, which included time for academic instruction and enrichment activities, helped students make “marginal progress”—progress that is meaningful, she said.

“This is some of the very first good news we’re getting on any COVID recovery,” Morton said. Still, she added, “layering supports is going to be essential. Students can’t just get summer school.”

Districts have used federal COVID relief funds to support summer programs, tutoring initiatives, and other academic recovery plans—money that runs out in September 2024.

But maintaining multiple options for students requires funding and resources, Morton said. “To be able to do that, they’re going to need support from states and even from the federal government.”

Summer programs show small gains in math

The summer school study focused on summer learning programs in eight school districts. All of these districts served higher percentages of Black and Hispanic students and higher percentages of students eligible for free- and reduced-price lunch than the national average.

The researchers compared students who were eligible to attend summer school and did attend to students who were eligible but did not attend. They controlled for students’ characteristics, including race and free- and reduced-price lunch status, and their spring 2022 achievement levels.

Students who attended summer programs scored higher on an assessment of their math skills than students who did not. The researchers found a small, but positive, effect of summer learning on math achievement—0.03 of a standard deviation, which they estimated would mitigate about 2-3 percent of the total learning loss experienced in those districts. Most of this improvement was driven by upper elementary students.

The researchers found no effect for summer school in reading.

The programs did help students get extra learning time—the researchers found that each day of summer school produced academic gains roughly equivalent to each day of a regular school year. But these gains only made a tiny dent in academic recovery writ large.

Programs in the study were between 15-20 days long. On average, only 25 percent of targeted students participated. “There’s an opportunity here to scale summer school more widely, and potentially to make it longer,” said Morton.

These kinds of changes would bring programs more in line with summer learning best practices, said Jodi Grant, the executive director of the Afterschool Alliance, an advocacy organization. Grant was not involved in the study.

“Even with intense support, it’s really hard to move the needle over a short period of time,” she said.

Grant also cited research-backed recommendations that summer learning should include opportunities for enrichment activities—not only academics. Doing so can make summer programs more attractive; they shouldn’t have to feel like a “punishment,” she said.

Conditions for effective tutoring programs

Designing an effective program is the first step. Implementing that program presents a whole new set of challenges.

This piece of the puzzle—implementation—is the focus on the working paper on tutoring.

“We know this intervention is really, really effective,” said Loeb. “But what hasn’t gotten nearly as much attention is that it’s not that easy to implement.”

Researchers reviewed 33 articles published since 2000. They found strong implementation relied in large part on the buy-in of principals and school leaders.

“Education leaders served as gatekeepers of student and staff time, school space, and data/documentation on students’ skills and instructional needs,” the researchers wrote.

See also

Teacher at a desk helping an elementary girl with her work.
E+/Getty

Having a dedicated and informed program manager was important, too, Loeb said. “You really need someone at the district level who is kind of organizing the approach, making sure that it aligns with what we can see from the results are the most promising practices,” she said.

Program managers need to be compensated for the work they’re doing, she added. That money could come from new funding, or districts could reprioritize funds. “But you definitely need somebody who has paid time to do it, and who is really responsible [for] it,” she said. “You can’t just add to people’s work without taking something away.”

Meeting conditions like these has been challenging for school districts.

The National Center for Education Statistics’ School Pulse Survey, an ongoing study of how schools are changing since the pandemic, polled school leaders in December 2022.

About a third of school leaders said they didn’t have the capacity to provide high-dosage tutoring to students who needed it. Two out of five schools said they couldn’t find time in their regular schedule for tutoring.

Still, Loeb said, better implementation is “not undoable.”

“Because it’s not easy,” she said, “we should be putting effort into thinking about the implementation process and really working on doing the implementation well.”

Events

Special Education K-12 Essentials Forum Innovative Approaches to Special Education
Join this free virtual event to explore innovations in the evolving landscape of special education.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
STEM Fusion: Empowering K-12 Education through Interdisciplinary Integration
Join our webinar to learn how integrating STEM with other subjects can revolutionize K-12 education & prepare students for the future.
Content provided by Project Lead The Way
School & District Management Webinar How Pensions Work: Why It Matters for K-12 Education
Panelists explain the fundamentals of teacher pension finances — how they are paid for, what drives their costs, and their impact on K-12 education.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Achievement Opinion How Winning the National Spelling Bee Prepared Me for High School
Studying for the bee was overwhelming at times, but the academic and emotional benefits have been invaluable, writes 9th grader Dev Shah.
Dev Shah
4 min read
 Lots arrows missed hitting target mark and only one hits the center. If at first you don't succeed, try again!
Vanessa Solis/Education Week + iStock/Getty Images
Student Achievement How Teachers Build Confidence to Motivate Middle Schoolers in STEM
Sixth through eighth grade marks a shift in what motivates students, presenting a big challenge for science and math teachers.
8 min read
Photo illustration of teen boy working with model.
F. Sheehan for Education Week + E+ / Getty
Student Achievement What the Research Says Want to Boost Students’ Focus During Tests? Check the Time of Day
New study finds distractions spikes when students take reading and math assessments after lunch
3 min read
Illustration of test items on a flat surface, including test booklet, test sheets, pencils, eraser and pencil sharpener.
DigitalVision Vectors + iStock/Getty + Education Week
Student Achievement Opinion The Trouble With 'Equitable Grading'
Equitable grading does in fact lead to lower standards, argue two critics of the practice.
6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty