A widely used, and initially successful, intervention for struggling beginning readers may hurt students’ reading growth in the long run, a new study finds.
Reading Recovery was considered one of the breakout stars of the federal Investing in Innovation program, after a massive randomized controlled study found the literacy program helped struggling 1st graders gain significant ground in reading. But new findings from a longitudinal follow-up of the program suggest that by 3rd and 4th grades, former Reading Recovery students performed significantly worse than their peers who did not participate in the program.
The results, presented at last week’s annual meeting of the American Educational Research Association, come as districts across the country search for ways to help catch up students who lost ground in reading during the pandemic. One-to-1 interventions like Reading Recovery have shown significant benefits in prior studies, but can be among the most expensive to maintain, both in training and staff time.
Reading Recovery, developed in the 1970s by New Zealand literacy researcher Marie Clay, is now used in Australia, Britain, and the United States as well.
Teachers provide 30-minute, 1-to-1 lessons with students who show early signs of reading difficulties. In each lesson, a child reads both familiar and new texts while a teacher keeps a “running record” of which words the student reads incorrectly, with notes about potential causes or miscues. The lessons also include writing and letter-sound practice.
In 2010, as part of the federal Investing in Innovation, or i3, research program, more than 8,000 of the lowest-performing readers were randomly assigned to participate in Reading Recovery either in the first or second half of their 1st grade year. That study found that after five months, children who participated in the program improved by more than 130 percent of the average reading growth for 1st graders nationwide—an effect considered at the time to be one of the largest seen for reading interventions.
While the initial i3 studies, conducted by the Consortium for Policy Research in Education at the University of Pennsylvania and the University of Delaware’s Center for Research in Education and Social Policy, could show causal evidence of the short-term benefits of the program, it did not allow researchers to understand the long-term effects because students in the control group also participated in Reading Recovery by the end of the year.
So, in a separate study, Henry May and his colleagues at the Center for Research in Education and Social Policy also tracked reading progress in 150 to 500 Reading Recovery schools each year from 2011-2015 and again in 2016-17. Each school included at least seven students who initially scored just below the reading cutoff to participate in Reading Recovery, as well as at least three students who performed just above the reading cutoff and could act as a control group.
For these students, the early benefits of Reading Recovery seemed to reverse themselves over time. Students who participated in the program in 1st grade had state reading test scores in 3rd and 4th grade that were roughly half a grade level below the scores of the students who had barely missed participating in Reading Recovery in 1st grade.
“It’s really important to do long-term impact studies because often they differ from short-term effects,” said Sean Reardon, a Stanford University researcher who is serving on the peer review committee for the Reading Recovery study. “Either the effects of the intervention compound over time with cycles of positive feedback, or they fade out over time with compensatory interventions” for students who were in the control group, he said.
“I think the big takeaway here is that the estimated long-term effects [of Reading Recovery] are negative, significant, and meaningfully large,” Reardon added.
What caused the drop?
What’s not clear is why Reading Recovery’s effects changed so dramatically over time. Some critics have argued that Reading Recovery’s focus on individual student errors can leave holes in explicit instruction for foundational skills. May said it’s also possible that the intervention improves early reading skills that don’t translate as well to skills needed for comprehension in later grades.
But he said students may also lose the ground they initially gain either because schools do not continue the same level of literacy supports in higher grades or because schools actively reprioritize literacy supports for students who show improvements because of early interventions.
The study also comes with major research caveats. For example, less than a quarter of the students could be tracked into 3rd grade, and only 15 percent could be tracked into 4th grade.. While the researchers did not find differences between the students who dropped out of the study in the treatment or control groups, the smaller sample could provide less information on the program as a whole.
In a written response to the study, Reading Recovery advocates argued that students who were prioritized for support while they were in Reading Recovery may have been given less support in later years. “Reading Recovery was not designed to be a panacea,” they wrote, arguing, “the intervention is successful with a majority of the lowest 1st grade readers and writers who receive a full series of lessons. These students are able to continue to make satisfactory progress with the support of good classroom instruction.”
Similar concerns have been raised in other high-profile longitudinal evaluations, including those of the federal Head Start program, for example, in which early supports were not sustained in later grades and early benefits faded out.
May cautioned that Reading Recovery still has more evidence of benefits in early grades than many other common reading interventions. “I would say it’s still perhaps the most effective intervention if you want to produce benefits in 1st grade,” he said. “But that said, when you look at overall impacts across grades ... I do have concerns” about long-term effects.
However, the results may change the equation for schools considering the program: In a connected cost-benefit analysis of 18 reading interventions, which was also presented at the research meeting, Reading Recovery was among the most expensive, ranging from about $5,400 to more than $10,000 per student, depending on the school.
