The Reading First Impact Study interim report released by IES last month upset a lot of the program’s fans, who’ve seen progress in their own schools/districts or on a statewide basis. It caused a bit of hand-wringing, and then a round of number crunching. Local and state representatives went to their databases and began printing off page after page of test results from Reading First schools, where they say there’s been dramatic improvements.
Some of those analyses, including this, were then tapped by bloggers, RF-friendly columnists, and national organizations to argue that the Impact Study—which found that the $1 billion-a-year investment in the program has had no effect on student’s reading comprehension—is flawed.
Russ Whitehurst, who heads the IES, said it is quite possible that schools and districts in the program have seen gains in student achievement. But the proponents are not measuring the same thing as the impact study, which used a comprehension measure and compared RF schools with those in the same districts that did not receive the grants.
Much of the data being rolled out to show RF’s impact, including a new 7-page paper by researchers at the Northwest Regional Educational Laboratory in Portland, Ore., is based on the DIBELS oral-reading-fluency test. (Analyses in Ohio and Idaho, linked above, are based on improvements on the state reading tests.) The DIBELS ORF test counts how many words in a passage students can accurately read in a minute. In many of the states using the test to gauge RF progress, 110 words is benchmark for a 3rd grader. The NWREL study found that in five states that the lab evaluates for RF—Alaska, Arizona, Montana, Washington, and Wyoming—there was “steady improvement” in the percentage of students meeting the benchmark, as well as a decrease in the proportion of 3rd graders falling well below benchmark.
There has been a lot of debate in the field, however, over just how useful the DIBELS is in gauging how students are doing. It does not test comprehension, the end goal of reading instruction, and while we want kids to read accurately, it is not clear that getting them to read quickly aloud is a worthy aim. It also is not intended as an outcome measure.
None of these local- or state-focused reports offers a complete or rigorous assessment of RF. The final impact study promises more: another year of comprehension data, a fluency measure, classroom observations, and study of the effect of certain teaching practices on achievement.
