The prescription was everywhere as the full extent of academic damage wrought by the pandemic first became clear: Set up tutoring programs to catch students up.
The same advice echoed from policy papers, think tanks, and the federal government, which put hundreds of billions into school recovery dollars. Experts and researchers heralded tutoring as an evidence-backed solution for addressing significant academic gaps, and some school systems saw early successes.
Now, though, several evaluations of pandemic-era tutoring programs are showing smaller effects on student progress than expected—or revealing that didn’t improve outcomes at all.
The most recent of these, from researchers at Stanford University’s SCALE Initiative, examined math and reading tutoring programs in a large, urban district during the 2022-23 and 2023-24 school years. Neither led to overall gains in academic achievement.
But when researchers dug deeper into the data, they identified implementation problems that could be driving these null effects.
The findings add to a growing body of evidence around tutoring in a post-COVID landscape that suggests the effectiveness of a program hinges on the nitty-gritty details of how it is run—how often students meet with their tutors, for instance, or whether lessons are tailored to their specific needs.
Studying these implementation details could help school systems build more effective tutoring initiatives in the long run, said Elizabeth Huffaker, an assistant professor of educational leadership and policy at the University of Florida, and the lead author on the SCALE paper.
“There is a lot of potential upside to findings like this, even though we wish that students were seeing more across-the-board gains as soon as possible,” she said.
Successful tutoring is hard to replicate at scale
The tutoring programs that districts stood up over the past five years are still in relatively early stages, said Matthew Kraft, a professor of education and economics at Brown University who has studied tutoring’s effectiveness.
“People should be asking the question, ‘Should we spend money on tutoring?’ That’s a totally fair question,” he said. But findings that some pandemic-era tutoring programs aren’t effective right now doesn’t preclude them becoming more successful as districts refine and retool implementation, he said.
“We haven’t gotten there,” said Kraft. “We are tinkering toward that possible equilibrium. I think it would be a real loss in incremental innovation and improvement if we abandon that possibility.”
From another vantage point, though, the problem is the same one that has continually plagued other major efforts to boost achievement, from class-size reduction to comprehensive school reform models to curriculum reform: scaling.
In practice, it’s often hard for schools to replicate the precise conditions that research on tutoring suggests can move the needle. Those include students learning from tutors one-on-one or in very small groups at least three times a week for about 50 hours of instruction over the course of a semester.
In a 2024 meta-analysis, Kraft and researchers Beth Schueler and Grace Falken examined 265 randomized controlled trials of tutoring. On average, these studies showed that tutoring had a large, positive effect on student achievement.
But when they reran the numbers looking only at the studies that tested large-scale tutoring programs in the United States, and measured students’ outcomes via standardized tests, the effect size shrank by about a half to two-thirds, with larger tutoring programs having smaller effect sizes.
In part, that’s because larger programs tend to have higher student-tutor ratios and less time spent in tutoring than smaller programs, Kraft said. It’s also possible, he said, that “the quality of delivery and implementation starts to slip when programs are scaling.”
It’s easier to find 10 strong tutors, in other words, than 100.
“What we’re learning from a really unbelievable wealth of rigorous studies, including the new study by SCALE, is a core insight in education,” Kraft said. “Implementation always trumps the base design of an educational intervention. You have to do something well for it to work.”
What went wrong in one kindergarten tutoring program
The SCALE study examined an early literacy tutoring program with 270 kindergartners and 1st graders, and an early numeracy program that served about 1,000 kindergartners. Paraprofessionals served as tutors in both programs.
Neither group of students received as much tutoring as the programs suggested. Students in the reading program, on average, went through about 30% of the recommended sessions, while students in the math program went through about 36% of their lessons.
In part, this was due to school testing schedules, said Huffaker. Tutoring couldn’t start until students were evaluated with initial assessments, which delayed the beginning of the program to November. And within the math program, almost half of the students who were supposed to receive tutoring never got it—administrators didn’t tell classroom teachers that those students were supposed to participate in the study.
But because the tutoring program was district-wide, the researchers were able to take advantage of differences in implementation in different schools. They found that when students received at least half of the sessions they were supposed to in the math program, achievement started to grow, and the effect got larger the more of the program that students attended.
The result delivers a strikingly simple insight, but one that could explain why some large-scale programs are found ineffective: “The core idea and promise of tutoring is showing up, but only if students are receiving the tutoring,” Huffaker said.
Dosage, tailoring both key factors to effective programs
In reading, the researchers found different patterns. Even when students completed more of the tutoring sessions, they didn’t see gains, on average.
Instead, lower-achieving students saw some growth compared to a control group, while students who started at a higher baseline didn’t.
The literacy program featured evidence-based instruction, but every student started at the same place, with very beginning skills in phonemic awareness—the ability to recognize and manipulate sounds. It’s likely, Huffaker said, that most students didn’t reach enough material to grow much beyond where they started.
The math program, by contrast, allowed for more individualized instruction. “There was less of a requirement to do every lesson in order, and more flexibility to skip around,” she said.
Together, the findings suggest the importance of tailoring tutoring to student needs, and making sure they have enough time with tutors to make progress, Huffaker said.
Through interviews with school leaders, the researchers learned that campuses where students progressed through more of the tutoring program and were more successful had prioritized running the program effectively.
“They instituted dedicated time every single day for the paraprofessional and the teacher to coordinate about instruction, and to make sure this is actually happening,” Huffaker said.
Leaders need to allot time and effort to this kind of collaboration, she added, especially if the programs depend on people who already wear a lot of hats in the school.
