Reading Recovery, a popular one-to-one tutoring program that Bush administration officials sought to shut out of a high-profile federal reading program, has gotten a rare thumbs-up from the federal What Works Clearinghouse.
The positive rating comes after prominent researchers and federal reading officials tried to dissuade states and districts from paying for Reading Recovery with funds from the $1 billion-a-year Reading First program, which calls on school systems to spend their grant money on programs backed by “scientifically based research.” In their objections to the tutoring program, critics raised questions about its cost and cited problems in the studies attesting to its effectiveness.
“I think this is good news for all the school superintendents who kept Reading Recovery alive in their schools,” said Jady Johnson, the executive director of the Reading Recovery Council of North America, a nonprofit group based in Worthington, Ohio. “I’m hoping this report will signal a change in direction for the [U.S. Education] Department.”
Read the clearinghouse report on the Reading Recovery program, posted by the What Works Clearinghouse.
Yet some of the program’s early critics said in interviews last week that many of their original concerns remained.
“I never said Reading Recovery is ineffective,” said Jack M. Fletcher, one of 32 researchers who signed a widely circulated 2002 letter critiquing the program. “The real issue with Reading Recovery is the idea that it has to be done individually, when there’s a substantial research base on small-group interventions that shows there’s no drop-off in effectiveness.”
Imported to the United States from New Zealand in 1984, Reading Recovery is an intensive, 12- to 20-week pullout program that targets the lowest-achieving 1st graders. Proponents estimate it has served more than 1.6 million students in this country.
In the What Works review, posted online March 20, the clearinghouse said the program had “positive” effects—the highest evidence rating possible—on students’ alphabetic skills and general reading achievement. The reviewers also determined that the program had “potentially positive” effects, its next-highest rating, on reading fluency and comprehension.
On the clearinghouse’s “improvement index,” a measure used to provide a common metric on program effects, researchers found that the average 1st grader who completed Reading Recovery could be expected to score 32 percentile points higher in general reading achievement than similar students not in the program.
Methods Questioned
That’s high praise from the clearinghouse, which the Department of Education’s Institute of Education Sciences created in 2002 to vet research on “what works” in education. So few education studies meet the clearinghouse’s tough research-quality criteria that some critics have dubbed it the “nothing works” clearinghouse. (“‘One Stop’ Research Shop Seen as Slow to Yield Views That Educators Can Use,” Sept. 27, 2006.)
Of the 78 studies the clearinghouse reviewed on Reading Recovery, for instance, only five passed muster at varying levels. Most of the five studies, which involved a total of 700 students, were experiments in which groups of students were randomly assigned to either a Reading Recovery group or a comparison group.
“Our job is not to weigh in on whether Reading First had the right curricula in the programs that districts have chosen,” said Phoebe H. Cottingham, the commissioner of the National Center for Education Evaluation and Regional Assistance, which oversees the clearinghouse. “We’re simply giving people research facts so they can decide on their own how much weight they want to put on the findings and make their own judgments.”
A short-term tutoring program intended to serve 1st graders in the bottom 20 percent in reading achievement. Pupils are chosen for the program by school staff members based on prior reading achievement, diagnostic testing, and teacher recommendations.
SOURCE: What Works Clearinghouse
She also said the clearinghouse plans to publish 20 more reports on reading interventions by July.
Nonetheless, Mr. Fletcher raised questions about the measures researchers used to track reading progress in the five studies on which the clearninghouse’s Reading Recovery report is based. The studies used a mix of standardized reading tests and a scale called the Clay Observation Survey, which was developed by Reading Recovery founder Marie Clay. Mr. Fletcher said the instrument may be biased toward what is taught in Reading Recovery lessons and may have other technical faults.
Those objections were countered last week by Richard L. Allington, an education professor at the University of Tennessee in Knoxville and a former president of the International Reading Association.
“I don’t think [the Clay Observation Scale is] any more of a concern than using DIBELS,” he said, referring to the Dynamic Indicators of Basic Early Literacy Skills, a test that is widely perceived to be the Bush administration’s favored measure for gauging students’ reading progress under Reading First. That test was devised by Reading First consultants and is being used to tout the federal initiative’s success.
“The question now is are we going to take all the interventions off the Reading First Web sites that don’t meet the What Works criteria,” Mr. Allington continued. “I don’t have a lot of confidence that anyone in Washington actually cares about the evidence.”
The Web sites feature reviews of reading series and programs that Reading First consultants directed states to for guidance on products that would pass muster for use in participating schools. The sites, at the University of Oregon and Florida State University, were not endorsed by the Education Department, but the consultants’ advice led to perceptions that they represented a federally approved list of research-based reading programs, the department’s inspector general found.
Complaints Backed
Reading Recovery proponents were among three groups that filed complaints in 2005 with the inspector general alleging that federal Reading First officials and consultants tried to steer states and districts toward reading programs and assessments that they favored—and, in some cases, had financial ties with—and away from other programs with substantial research track records.
The inspector general largely substantiated those complaints in a series of reports issued over the past five months as part of a broad review of the Reading First program, which was launched five years ago as part of the federal No Child Left Behind Act.
For instance, the first of those six reports found, then-Reading First Director Christopher J. Doherty, in a conference call in late 2002 with six Kentucky school officials, told the group that Reading Recovery was “not scientifically based.”
The report also noted that Susan B. Neuman, who was assistant secretary for elementary and secondary education at the Education Department during the grant-application process for Reading First, advised Mr. Doherty in January 2002 that the language in the program guidelines should not encourage the use of Reading Recovery.
Mr. Doherty and Ms. Neuman have since left the department.
According to an Education Week review of e-mail documents, federal officials repeatedly discussed Reading Recovery, and their desire to prevent states from allowing use of the program in Reading First schools, during 2002 and 2003, when states were submitting their grant proposals.
Elaine Quesinberry, an Education Department spokeswoman, said last week that the department “is prohibited from endorsing any particular curriculum, and we will not encourage states to use any one curriculum over another.” Ms. Neuman said in an interview that the instructions she relayed to Mr. Doherty were not to discredit Reading Recovery, but only to ensure that Reading First funding went mostly to comprehensive core reading programs, not pullout programs that benefitted only a handful of students.
According to Reading Recovery’s Ms. Johnson, the negative publicity and the department’s effort hurt the program’s popularity. The number of students participating, she said, dropped from 159,000 a year in 2002, the first year of Reading First, to 109,000 four years later.
Cost-Effective?
G. Reid Lyon, an influential adviser to the department on reading issues at the time and a critic of Reading Recovery, did not dispute the What Works review.
“The evidence is what it says,” said Mr. Lyon, who headed the reading-research branch of the National Institute for Child Health and Human Development until 2005. “So the question we need to ask is what level of professional development is needed to implement a program with fidelity? Can the district provide that? Can we cover the expense of the program? Is it cost-effective?”
Some experts consider the program costly because it requires one-to-one tutoring and because the teachers who supervise the program in their schools must undergo a year of university-based training. Reading Recovery advocates, however, contend those costs are largely offset by the savings that come as fewer children are referred to special education.
Critics have also noted that most of the studies were conducted by researchers affiliated with Reading Recovery, which is not unusual among the studies the clearinghouse reviews.
Alan E. Farstrup, the executive director of the Newark, Del.-based International Reading Association, said debates over the best methods for teaching reading miss the point. “There are no best methods, but there are best matches between the needs of individual children and evidence-based practices,” he said. “I would call this report useful evidence, so that there’s a broader array of programs to choose from.”
