Tutoring is the top-billed remedy to help students make up for disrupted learning. States and districts are spending millions in federal funds to pay for it. But can it work online when so many other efforts to move instruction online have fallen short?
A new study from a team of researchers is the very first to test the hypothesis using a randomized experiment. And the results, while far from a silver bullet, show some promise and suggest some lessons for other online tutoring efforts.
Probably because most students in the study received only around three hours of tutoring in a 12-week period—far short of the intensity prescribed in prior research—the findings weren’t strong enough to be statistically significant.
But they pointed in a positive direction overall, and the study found suggestive evidence that more tutoring time would have yielded stronger results. The study also relied on volunteer college students to serve as the tutors, a model that can offer some cost savings.
“It’s not a huge amount of tutoring overall, and in some ways, despite the lack of statistical significance, I’m somewhat even surprised that we found positive and suggestive effects given the basically low dosage,” said Matthew Kraft, an associate professor of education and economics at Brown University and one of the team of researchers who conducted the study.
Previous studies bolster tutoring—but not in a pandemic
A wide body of prior research points to intensive tutoring, in one-on-one or very small groups, as an effective way to boost learning. In the early days of the pandemic, researchers, scholars, and media outlets highlighted it as a promising approach to support students who were struggling academically.
But that research had limitations, too. All of it predated the pandemic. Much of it was done in person, with qualified tutors, and embedded for lengthy periods in the school day.
Looking to cut to the chase? Here’s what district leaders should take away from new research on online tutoring.
- Using college students to tutor middle schoolers in math and reading online seemed to improve learning, but the study can’t say that for sure because of the limited amount of tutoring time.
- More tutoring time and a longer program likely would produce better results.
- College volunteers can help bring the cost of tutoring programs down, but probably can’t make up an entire tutoring corps.
- There’s still a lot to learn about how to make virtual tutoring work well, such as tutor selection, training, and professional development.
Those conditions are far different from the reality district leaders have been forced to contend with the last two years—exhausted staff, a labor market in upheaval, virtual learning. They’ve had to turn their constraints around like a Rubik’s cube puzzle, trying to get the pieces to land the right way up.
What if schools use certified, trained teachers? Costs rise—if you can find and hire them in the first place. Use volunteers or college students? Cheaper, but they typically have less experience with teaching methods. Reduce the amount of tutoring? You may not get as big a bang for your buck. Try virtual tutoring? So far, the best evidence comes from a pilot program—in Italy, not the United States.
The new study comes much closer to the realities on the ground facing districts.
Kraft and his team worked with a nonprofit begun by college students, CovEd, which provided free tutoring services beginning in the 2020-21 school year.
Some 230 tutors worked with 6th through 8th graders at a Chicago middle school. Half the students were assigned to tutors who aimed to provide about an hour’s worth of tutoring each week. The other students went to their regular advisory period.
Even though the tutoring was provided virtually, CovEd and the team faced challenges sustaining the program for the duration of the study.
The study began in spring 2021, just as some college students were returning to their own in-person courses. In summer, volunteers’ schedules changed as they picked up jobs. By fall, the program struggled to find enough tutors to keep it going. Absenteeism was an issue too, though not due to any one factor, said Evelyn Wong, a manager of CovEd who helped found the nonprofit.
“Another challenge was students who didn’t have reliable internet access or whose parents were working, so the fact that the school had built the tutoring into their schedule was really helpful,” she said. “Even then, a lot of students have things going on at home. The mentor would show up and students wouldn’t be able to [participate], because they were looking after a younger sibling, and the opposite was true, too; a mentor would have a job come up.”
It was sobering for the researchers, too.
“In my experience trying to implement, and talking to district leaders trying to scale tutoring, uniformly they’ll tell you there are substantial implementation challenges,” Kraft said. “That shouldn’t be surprising given there’s a general labor shortage for people working in the education sector …. But even beyond that, changing ossified school schedules and structures in dynamic and new ways [to make space for tutoring] takes pushing back against kind of the existing norms, and that isn’t always easy.”
The resulting test scores, while positive, weren’t large enough to rule them out as being a product of test error. But the study did show that those students who had more opportunities to attend tutoring sessions seemed, overall, to progress more than those who had fewer.
Logistical challenges are real—but not insurmountable
Some of these logistical challenges exist across the virtual tutoring field, said Shaan Akbar, the co-founder of Tutored by Teachers, a company that uses certified teachers to provide online tutoring services to schools and districts.
Akbar, who was not involved in the study, said such programs are harder to set up when harried administrators—and even the procurement officials who work out the tutoring contracts—are busy covering classes in schools thanks to staff shortages and teachers being ill with COVID.
“Yes, they want to work with you. Yes, they signed up with you. But man, they have to get you the student data and schedule you into their day—it’s school day design stuff,” he said. “And you have to have staff on the ground to monitor those kids or usher them to the computer lab and monitor them. And so what we’re finding is that we are more often than not deploying people of our own to the schools to support all of that.”
All that said, he said, his organization’s internal data shows progress among students when they receive at least an hour and a half of tutoring each week. (That’s far more than students in the study received.)
“The low cost, on-demand tutoring, the quick-solve stuff, isn’t going to yield the outcomes you want,” Akbar said. “You have to find deep partnerships and do it right to get the outcomes you need.”
And matching students to tutors is an art, not a science, Wong added.
“It’s about bringing a sense of normalcy,” she said. “Having a mentor they can connect with and look up to is more than 80-90 percent of the challenge here, rather than just looking for a match between mentors and subject areas.”
Thus the new study is really a beginning not an end—and begs for follow up.
“We have very few data points on the effectiveness of online tutoring. This is a new data point,” Kraft said. “It is far from the definitive answer, and we need lots of dots to try to see the whole picture. We’ve filled in one dot.”
