Federal Reading Review Overlooks Popular Texts

By Kathleen Kennedy Manzo — August 28, 2007 4 min read
  • Save to favorites
  • Print

When a federal review of beginning-reading programs was commissioned four years ago, experts and educators hoped it would help school leaders sift through the vast marketplace of instructional materials and find those most effective for improving achievement. But the long-awaited study by the What Works Clearinghouse, released this month, may not fully deliver on that promise.

In fact, the analysis found that few comprehensive or supplemental programs have empirical proof that they work. And none of the most popular commercial programs on the market—including McGraw-Hill’s Open Court, Scott Foresman Reading, and Houghton Mifflin Reading, which have earned hundreds of millions of dollars in sales to districts—had sufficiently rigorous studies to be included in the review by the clearinghouse.

“They tended not to have studies with randomized-control trials or with experimental designs that met the clearinghouse’s evidence standards,” said Jill Constantine, the principal director of the review.

Most of the programs deemed to have “positive effects” or “potentially positive effects” in the review were supplemental or intervention programs, not core reading series. In the review of nearly 900 studies, just 51 met the standard for evidence, meaning just one or two for each program.

More than 120 other programs, however, had no studies that met the requirements.

The complexity of the study, and the scarcity of suitable research on the topic, point to the difficulty of determining what works in the classroom, said James W. Kohlmoos, the president of the Knowledge Alliance, a Washington-based nonprofit organization that works with education groups to translate research findings to instruction.

“In the very early stages, we thought it was something that could be of value, but it’s only of marginal value right now,” Mr. Kohlmoos said of the federal review. “We all wish there was more in there and the results were more definitive,” he said, “but it just shows that this is a far more complex and difficult effort than we all originally thought.”

Smaller Programs Reviewed

The clearinghouse, which the U.S. Department of Education’s Institute of Education Sciences set up in 2002 to vet research on “what works” in education, has given few of its coveted positive ratings.

What Works: Effectiveness of Reading Programs

Source: Institute of Education Sciences


Just one of the reading programs examined was found to have positive effects or potentially positive effects across all four of the domains in the review: alphabetics, fluency, comprehension, and general reading achievement. That program, Reading Recovery, involving intensive, one-on-one tutoring, has drawn criticism over the past few years from prominent researchers and federal officials who claim it is costly and lacks evidence that it helps struggling readers.

Success for All, a whole-school-reform program developed at Johns Hopkins University in Baltimore, got the favorable rating on alphabetics and general reading achievement, but mixed results on comprehension. Voyager Universal Literacy System, published by the Dallas-based Voyager Learning, was found to have potentially positive effects on alphabetics but potentially negative effects on comprehension. Accelerated Reader, distributed by Renaissance Learning Inc., was found to have a potentially positive impact on comprehension and general reading achievement.

The Success for All review is not as glowing as previous federal and private studies, which have put it in an exclusive group of programs that meet the highest standard of evidence that they are effective at teaching children to read. (“Long-Awaited Study Shows ‘Success for All’ Gains,” May 11, 2005.)

Robert E. Slavin, the founder of Success for All, said the clearinghouse review discounted such studies, which were based on large numbers of students exposed to the whole-school-reform program, and instead favored research on targeted interventions for small groups of students.

Those products—such as Start Making a Reader Today, Kaplan SpellRead, and Peer-Assisted Learning Strategies, or PALS—also got positive reviews.

But Mr. Slavin said the review is missing important information on the large commercial programs.

“The What Works Clearinghouse was designed to ... tell educators in a fair, nonpolitical, scientifically justifiable way what is the strength of evidence of various programs,” he said. “How could it not have done so on the largest programs? To go for five years without getting that job done boggles the imagination.”

In fact, the clearinghouse report lists 36 products, including some widely used series, that are in the process of being reviewed, but notes that each has just one study that meets the criteria “with reservations.”

First Appraisal

Tom Stanton, a spokesman for the New York City-based McGraw-Hill, wrote in an e-mail that the company’s programs have been developed over decades of “intense study, scientific research, and field testing.”

“The key question,” he added, “is, ‘do our programs drive student achievement?’ Our answer is that they do, and that we have results to prove it.”

The reading review, which has been under way for more than three years, is the first in an ongoing appraisal of reading programs, according to Phoebe H. Cottingham, the commissioner of the National Center for Education Evaluation and Regional Assistance, which oversees the clearinghouse.

Related Tags:


Student Well-Being K-12 Essentials Forum Social-Emotional Learning: Making It Meaningful
Join us for this event with educators and experts on the damage the pandemic did to academic and social and emotional well-being.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Privacy & Security Webinar
K-12 Cybersecurity in the Real World: Lessons Learned & How to Protect Your School
Gain an expert understanding of how school districts can improve their cyber resilience and get ahead of cybersecurity challenges and threats.
Content provided by Microsoft
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Trauma-Informed Schools 101: Best Practices & Key Benefits
Learn how to develop a coordinated plan of action for addressing student trauma and
fostering supportive, healthy environments.
Content provided by Crisis Prevention Institute

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Conservative Backlash Pushes Biden Administration to Dissolve New National Parent Council
Parent advocacy groups sued the U.S. Department of Education over the council, claiming it was unlawfully biased.
6 min read
U.S. Secretary of Education Miguel Cardona talks during a roundtable with School District of Philadelphia officials, the principal, a teacher, and a parent at the Olney Elementary School Annex in North Philadelphia on Tuesday, April 6, 2021.
U.S. Secretary of Education Miguel Cardona talks during a roundtable discussion last year in Philadelphia.
Tim Tai/The Philadelphia Inquirer via AP
Federal How a Divided Congress Will Influence K-12 Education Policy
The U.S. Senate and House of Representatives education committees will have new leaders in January.
6 min read
Sen. Bernie Sanders, I-Vt., speaks Monday, June 13, 2022, during a debate with Sen. Lindsey Graham, R-S.C, Hosted by Fox News at the The Edward M. Kennedy Institute in Boston for a debate intended to prove that bipartisanship isn't dead.
Sen. Bernie Sanders, I-Vt., speaks during a June debate with Sen. Lindsey Graham, R-S.C, at The Edward M. Kennedy Institute in Boston. Sanders is poised to become the chair of the Senate Health, Education, Labor, and Pensions Committee.
Josh Reynolds/AP
Federal What the Federal 'Don't Say Gay' Bill Actually Says
The bill would restrict federal funds for lessons on LGBTQ identities. The outcome of this week's election could revive its prospects.
4 min read
Demonstrators gather on the steps of the Florida Historic Capitol Museum in front of the Florida State Capitol on March 7, 2022, in Tallahassee, Fla. Florida House Republicans advanced a bill, dubbed by opponents as the "Don't Say Gay" bill, to forbid discussions of sexual orientation and gender identity in schools, rejecting criticism from Democrats who said the proposal demonizes LGBTQ people.
Demonstrators gather on the steps of the Florida Historic Capitol Museum in Tallahassee on March 7, 2022. Florida's "Don't Say Gay" law was a model for a federal bill introduced last month.
Wilfredo Lee/AP
Federal Fed's Education Research Board Is Back. Here's Why That Matters
Defunct for years, the National Board for Education Sciences has new members and new priorities.
2 min read
Image of a conference table.