Federal

Federal Reading Review Overlooks Popular Texts

By Kathleen Kennedy Manzo — August 28, 2007 4 min read
  • Save to favorites
  • Print

When a federal review of beginning-reading programs was commissioned four years ago, experts and educators hoped it would help school leaders sift through the vast marketplace of instructional materials and find those most effective for improving achievement. But the long-awaited study by the What Works Clearinghouse, released this month, may not fully deliver on that promise.

In fact, the analysis found that few comprehensive or supplemental programs have empirical proof that they work. And none of the most popular commercial programs on the market—including McGraw-Hill’s Open Court, Scott Foresman Reading, and Houghton Mifflin Reading, which have earned hundreds of millions of dollars in sales to districts—had sufficiently rigorous studies to be included in the review by the clearinghouse.

“They tended not to have studies with randomized-control trials or with experimental designs that met the clearinghouse’s evidence standards,” said Jill Constantine, the principal director of the review.

Most of the programs deemed to have “positive effects” or “potentially positive effects” in the review were supplemental or intervention programs, not core reading series. In the review of nearly 900 studies, just 51 met the standard for evidence, meaning just one or two for each program.

More than 120 other programs, however, had no studies that met the requirements.

The complexity of the study, and the scarcity of suitable research on the topic, point to the difficulty of determining what works in the classroom, said James W. Kohlmoos, the president of the Knowledge Alliance, a Washington-based nonprofit organization that works with education groups to translate research findings to instruction.

“In the very early stages, we thought it was something that could be of value, but it’s only of marginal value right now,” Mr. Kohlmoos said of the federal review. “We all wish there was more in there and the results were more definitive,” he said, “but it just shows that this is a far more complex and difficult effort than we all originally thought.”

Smaller Programs Reviewed

The clearinghouse, which the U.S. Department of Education’s Institute of Education Sciences set up in 2002 to vet research on “what works” in education, has given few of its coveted positive ratings.

What Works: Effectiveness of Reading Programs

Source: Institute of Education Sciences

BRIC ARCHIVE

Just one of the reading programs examined was found to have positive effects or potentially positive effects across all four of the domains in the review: alphabetics, fluency, comprehension, and general reading achievement. That program, Reading Recovery, involving intensive, one-on-one tutoring, has drawn criticism over the past few years from prominent researchers and federal officials who claim it is costly and lacks evidence that it helps struggling readers.

Success for All, a whole-school-reform program developed at Johns Hopkins University in Baltimore, got the favorable rating on alphabetics and general reading achievement, but mixed results on comprehension. Voyager Universal Literacy System, published by the Dallas-based Voyager Learning, was found to have potentially positive effects on alphabetics but potentially negative effects on comprehension. Accelerated Reader, distributed by Renaissance Learning Inc., was found to have a potentially positive impact on comprehension and general reading achievement.

The Success for All review is not as glowing as previous federal and private studies, which have put it in an exclusive group of programs that meet the highest standard of evidence that they are effective at teaching children to read. (“Long-Awaited Study Shows ‘Success for All’ Gains,” May 11, 2005.)

Robert E. Slavin, the founder of Success for All, said the clearinghouse review discounted such studies, which were based on large numbers of students exposed to the whole-school-reform program, and instead favored research on targeted interventions for small groups of students.

Those products—such as Start Making a Reader Today, Kaplan SpellRead, and Peer-Assisted Learning Strategies, or PALS—also got positive reviews.

But Mr. Slavin said the review is missing important information on the large commercial programs.

“The What Works Clearinghouse was designed to ... tell educators in a fair, nonpolitical, scientifically justifiable way what is the strength of evidence of various programs,” he said. “How could it not have done so on the largest programs? To go for five years without getting that job done boggles the imagination.”

In fact, the clearinghouse report lists 36 products, including some widely used series, that are in the process of being reviewed, but notes that each has just one study that meets the criteria “with reservations.”

First Appraisal

Tom Stanton, a spokesman for the New York City-based McGraw-Hill, wrote in an e-mail that the company’s programs have been developed over decades of “intense study, scientific research, and field testing.”

“The key question,” he added, “is, ‘do our programs drive student achievement?’ Our answer is that they do, and that we have results to prove it.”

The reading review, which has been under way for more than three years, is the first in an ongoing appraisal of reading programs, according to Phoebe H. Cottingham, the commissioner of the National Center for Education Evaluation and Regional Assistance, which oversees the clearinghouse.

Related Tags:

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Education Dept. Sees Small Cut in Funding Package That Averted Government Shutdown
The Education Department will see a reduction even as the funding package provides for small increases to key K-12 programs.
3 min read
President Joe Biden delivers a speech about healthcare at an event in Raleigh, N.C., on March 26, 2024.
President Joe Biden delivers a speech about health care at an event in Raleigh, N.C., on March 26. Biden signed a funding package into law over the weekend that keeps the federal government open through September but includes a slight decrease in the Education Department's budget.
Matt Kelley/AP
Federal What Would Happen to K-12 in a 2nd Trump Term? A Detailed Policy Agenda Offers Clues
A conservative policy agenda could offer the clearest view yet of K-12 education in a second Trump term.
8 min read
Republican presidential candidate and former President Donald Trump speaks at a campaign rally, March 9, 2024, in Rome Ga.
Former President Donald Trump speaks at a campaign rally, March 9, 2024, in Rome, Ga. Allies of the former president have assembled a detailed policy agenda for every corner of the federal government with the idea that it would be ready for a conservative president to use at the start of a new term next year.
Mike Stewart/AP
Federal Opinion Reading Scores Have Plunged Since the Pandemic. What This Senator Wants to Do About That
How can the nation support literacy efforts? The ranking Republican on the Senate education committee wants to hear from educators.
6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Federal Biden Calls for Teacher Pay Raises, Expanded Pre-K in State of the Union
President Joe Biden highlighted a number of his education priorities in a high-stakes speech as he seeks a second term.
5 min read
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
Shawn Thew/Pool via AP