Federal

Supplementary Reading Programs Found Ineffective

By Mary Ann Zehr — May 05, 2009 3 min read
  • Save to favorites
  • Print

Includes updates and/or revisions.

A federal study intended to provide insight on the effectiveness of supplementary programs for reading comprehension found that three such programs had no positive impact, while a fourth had a negative effect on student achievement.

In other words, the report concludes that none of the four programs studied—Project CRISS, ReadAbout, Read for Real, and Reading for Knowledge—is effective.

The large-scale randomized controlled study involved 6,350 students, who were all 5th graders, and 268 teachers in 10 urban districts with large numbers of disadvantaged students. The 89 schools in the study were randomly assigned either to use one of the reading curricula being reviewed or to a control group.

“It’s very distressing how hard it is to get bumps in the reading performance of students who are below grade level in reading after the primary grades,” said Catherine E. Snow, a professor of education at the Harvard Graduate School of Education, in response to the study. “You do an intervention with a 2nd grader, you’re changing direction on a speedboat, but when you do an intervention with a 5th grader, you’re changing direction on an oil tanker,” said Ms. Snow, a reading expert.

The findings are particularly distressing because the reading programs studied aren’t “just phonological-awareness programs,” she added, but rather seem to embody the right components for boosting reading comprehension.

‘Real Teachers, Real Schools’

Researchers for the study—conducted by Mathematica Policy Research Inc., for the U.S. Department of Education’s Institute of Education Sciences—used a general reading test, the Group Reading Assessment and Diagnostic Evaluation, and reading-comprehension tests of science and social studies to measure student achievement. In addition, they factored in students’ composite scores for all tests.

They concluded that Project CRISS, developed by Creating Independence Through Student-Owned Strategies; ReadAbout, produced by Scholastic Inc., and Read for Real, created by Chapman University and Zaner-Bloser, had no effect on reading comprehension. All three programs are already marketed to and used by schools. In addition, the researchers found that Reading for Knowledge, a program adapted for the study from a reading program created and marketed by the Success for All Foundation, had a negative impact on the composite test scores and the science-comprehension test scores for students using that curriculum.

Susanne James-Burdumy, the project director for the study for Mathematica, said the mission of such a study is to “find out what works for real teachers in real schools,” which may be different from how the reading interventions held up in the “highly controlled settings” in which they were initially tested.

Conclusions Disputed

Robert E. Slavin, a researcher and the founder of the Baltimore-based Success for All Foundation, dismissed the study’s conclusions. In an e-mail message, he contended that “IES-sponsored evaluations repeatedly evaluate programs by imposing them on teachers and school leaders who are not interested in them and are likely to implement them haphazardly, if at all, and then find, over and over again, that nothing works.”

But Phoebe Cottingham, the commissioner for the National Center for Education Evaluation and Regional Assistance at the federal agency, said “IES has never imposed on any teachers or schools programs, curricula, or new initiatives without their full cooperation.”

Mathematica worked hard during the recruiting stage to attract districts in which administrators and teachers were excited about participating, Ms. James-Burdumy said. Participation means free supplementary materials and professional development for schools. But, she added, not all educators in school districts may agree with the decision to take part.

Ms. James-Burdumy said the study found that 81 percent to 91 percent of teachers in the treatment groups reported they used the supplementary reading materials. Classroom observations revealed that across all four reading programs, teachers implemented 58 percent to 78 percent of the recommended practices.

Mr. Slavin also criticized the study for implementing the reading programs only in the 5th grade, which, he said, meant teachers had little support from colleagues in their schools to carry them out.

Harvard’s Ms. Snow noted that the findings of the study could be a reflection of the kinds of schools that participated more than the quality of the programs. “Highly challenged schools and districts don’t do a very good job of implementing anything,” she said.

A version of this article appeared in the May 13, 2009 edition of Education Week

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and K-12 education jubs at the EdWeek Top School Jobs virtual career fair.
Mathematics K-12 Essentials Forum Helping Students Succeed in Math
Student Well-Being Live Online Discussion A Seat at the Table: The Power of Emotion Regulation to Drive K-12 Academic Performance and Wellbeing
Wish you could handle emotions better? Learn practical strategies with researcher Marc Brackett and host Peter DeWitt.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Opinion A D.C. Insider Explains What’s Changed in Education Policy
The biggest thing that people don’t understand about federal education policy? How much the details really matter.
7 min read
The United States Capitol building as a bookcase filled with red, white, and blue policy books in a Washington DC landscape.
Luca D'Urbino for Education Week
Federal What Superintendents Think About a Steady Clip of Federal K-12 Changes
A state superintendent and two district leaders shared their thoughts on the latest changes coming from Washington.
4 min read
From left, Quentin J. Lee, superintendent of Talladega City Schools, Keith Konyk, superintendent of Elizabeth Forward School District, and Eric Mackey, Alabama's state superintendent of education, discuss the latest K-12 policy changes at the ISTELive 25 + ASCD Annual Conference 25 on July 2, 2025.
From left, Quentin J. Lee, superintendent of Talladega City Schools in Alabama; Keith Konyk, superintendent of Elizabeth Forward School District in Pennsylvania; and Eric Mackey, Alabama's state superintendent of education, discuss the latest K-12 policy changes at the ISTELive 25 + ASCD Annual Conference 25 on July 2, 2025.
Kaylee Domzalski/Education Week
Federal The Senate Passed a Federal Voucher Program. What's in It?
The measure would create a national program funding private school tuition through tax credits, though states would have to opt in.
7 min read
The Senate side of the Capitol is seen in Washington, early Monday, June 30, 2025, as Republicans plan to begin a final push to advance President Donald Trump's big tax breaks and spending cuts package.
The Senate side of the Capitol is seen in Washington early on June 30, 2025, hours before Republicans narrowly passed President Donald Trump's big tax breaks and spending cuts package. The bill includes the first major federal private school choice program.
J. Scott Applewhite/AP
Federal Senate Narrowly Passes Trump’s Big Tax Breaks and Spending Cuts Bill
The bill goes back to the House, where Speaker Mike Johnson warned off big revisions from his chamber’s version.
5 min read
Senate Majority Leader John Thune, R-S.D., center, joined from left by, Sen. John Barrasso, R-Wyo., the GOP whip, Finance Committee Chairman Mike Crapo, R-Idaho, and Budget Committee Chairman Lindsey Graham, R-S.C., speaks to reporters after passage of the budget reconciliation package of President Donald Trump's signature bill of big tax breaks and spending cuts, at the U.S. Capitol in Washington, D.C., on July 1, 2025.
Senate Majority Leader John Thune, R-S.D., center, joined from left by Sen. John Barrasso, R-Wyo., the GOP whip; Finance Committee Chairman Mike Crapo, R-Idaho; and Budget Committee Chairman Lindsey Graham, R-S.C., speaks to reporters after passage of the budget reconciliation package of President Donald Trump's signature bill of big tax breaks and spending cuts, at the U.S. Capitol on July 1, 2025. The bill includes the first major federal investment in a private school choice program.
J. Scott Applewhite/AP