When the interim report of the Reading First Impact Study came out in May, there was an uproar from fans of the federal program who said the methodology was seriously flawed. As I reported here, they cited the likely “contamination” of the comparison schools, meaning that even though they didn’t receive the grant money they were implementing many of the same policies and practices as participating schools. Given that many of the RF and non-RF schools were potentially benefiting from the same professional development, instructional materials, and practices, the critics argued, it would be surprising if the evaluation found much of a difference in achievement among the students.
With the release of the final impact study this week, the same argument is being made to temper the disappointing findings. But during the media briefing held Tuesday to explain all the findings to reporters, IES director Russ Whitehurst, and project director Beth Gamse, made a point of stressing the quality of the study.
“The study fulfills the legistlative mandate for a rigorous, scientifically valid and quantitative evaluation of Reading First,” Gamse said. The “quite large” sample of more than 30,000 students, “makes this one of the largest and also one of the most methodologically rigorous studies that this department has ever undertaken.”
Whitehurst said that contamination is a non-issue, given that the classroom observations and surveys show that reading instruction in the classrooms in RF schools is significantly different than in the comparison schools.
Even without this complication, however, neither group turned in a particularly stellar performance. There was no significant improvement in the scale scores, or in the percentage of students meeting proficiency, in 2nd or 3rd grades between 2005 and 2007, for either the RF or non-RF schools.
