Federal

Federal Reading Review Overlooks Popular Texts

By Kathleen Kennedy Manzo — August 28, 2007 4 min read
  • Save to favorites
  • Print

When a federal review of beginning-reading programs was commissioned four years ago, experts and educators hoped it would help school leaders sift through the vast marketplace of instructional materials and find those most effective for improving achievement. But the long-awaited study by the What Works Clearinghouse, released this month, may not fully deliver on that promise.

In fact, the analysis found that few comprehensive or supplemental programs have empirical proof that they work. And none of the most popular commercial programs on the market—including McGraw-Hill’s Open Court, Scott Foresman Reading, and Houghton Mifflin Reading, which have earned hundreds of millions of dollars in sales to districts—had sufficiently rigorous studies to be included in the review by the clearinghouse.

“They tended not to have studies with randomized-control trials or with experimental designs that met the clearinghouse’s evidence standards,” said Jill Constantine, the principal director of the review.

Most of the programs deemed to have “positive effects” or “potentially positive effects” in the review were supplemental or intervention programs, not core reading series. In the review of nearly 900 studies, just 51 met the standard for evidence, meaning just one or two for each program.

More than 120 other programs, however, had no studies that met the requirements.

The complexity of the study, and the scarcity of suitable research on the topic, point to the difficulty of determining what works in the classroom, said James W. Kohlmoos, the president of the Knowledge Alliance, a Washington-based nonprofit organization that works with education groups to translate research findings to instruction.

“In the very early stages, we thought it was something that could be of value, but it’s only of marginal value right now,” Mr. Kohlmoos said of the federal review. “We all wish there was more in there and the results were more definitive,” he said, “but it just shows that this is a far more complex and difficult effort than we all originally thought.”

Smaller Programs Reviewed

The clearinghouse, which the U.S. Department of Education’s Institute of Education Sciences set up in 2002 to vet research on “what works” in education, has given few of its coveted positive ratings.

What Works: Effectiveness of Reading Programs

Source: Institute of Education Sciences

BRIC ARCHIVE

Just one of the reading programs examined was found to have positive effects or potentially positive effects across all four of the domains in the review: alphabetics, fluency, comprehension, and general reading achievement. That program, Reading Recovery, involving intensive, one-on-one tutoring, has drawn criticism over the past few years from prominent researchers and federal officials who claim it is costly and lacks evidence that it helps struggling readers.

Success for All, a whole-school-reform program developed at Johns Hopkins University in Baltimore, got the favorable rating on alphabetics and general reading achievement, but mixed results on comprehension. Voyager Universal Literacy System, published by the Dallas-based Voyager Learning, was found to have potentially positive effects on alphabetics but potentially negative effects on comprehension. Accelerated Reader, distributed by Renaissance Learning Inc., was found to have a potentially positive impact on comprehension and general reading achievement.

The Success for All review is not as glowing as previous federal and private studies, which have put it in an exclusive group of programs that meet the highest standard of evidence that they are effective at teaching children to read. (“Long-Awaited Study Shows ‘Success for All’ Gains,” May 11, 2005.)

Robert E. Slavin, the founder of Success for All, said the clearinghouse review discounted such studies, which were based on large numbers of students exposed to the whole-school-reform program, and instead favored research on targeted interventions for small groups of students.

Those products—such as Start Making a Reader Today, Kaplan SpellRead, and Peer-Assisted Learning Strategies, or PALS—also got positive reviews.

But Mr. Slavin said the review is missing important information on the large commercial programs.

“The What Works Clearinghouse was designed to ... tell educators in a fair, nonpolitical, scientifically justifiable way what is the strength of evidence of various programs,” he said. “How could it not have done so on the largest programs? To go for five years without getting that job done boggles the imagination.”

In fact, the clearinghouse report lists 36 products, including some widely used series, that are in the process of being reviewed, but notes that each has just one study that meets the criteria “with reservations.”

First Appraisal

Tom Stanton, a spokesman for the New York City-based McGraw-Hill, wrote in an e-mail that the company’s programs have been developed over decades of “intense study, scientific research, and field testing.”

“The key question,” he added, “is, ‘do our programs drive student achievement?’ Our answer is that they do, and that we have results to prove it.”

The reading review, which has been under way for more than three years, is the first in an ongoing appraisal of reading programs, according to Phoebe H. Cottingham, the commissioner of the National Center for Education Evaluation and Regional Assistance, which oversees the clearinghouse.

Related Tags:

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Leadership in Education: Building Collaborative Teams and Driving Innovation
Learn strategies to build strong teams, foster innovation, & drive student success.
Content provided by Follett Learning
School & District Management K-12 Essentials Forum Principals, Lead Stronger in the New School Year
Join this free virtual event for a deep dive on the skills and motivation you need to put your best foot forward in the new year.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Modern Data Protection & Privacy in Education
Explore the modern landscape of data loss prevention in education and learn actionable strategies to protect sensitive data.
Content provided by  Symantec & Carahsoft

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Trump's VP Pick: What We Know About J.D. Vance's Record on Education
Two days after a gunman tried to assassinate him, former President Donald Trump announced Ohio Sen. J.D. Vance as his running mate.
4 min read
Sen. J.D. Vance, R-Ohio, right, points toward Republican presidential candidate former President Donald Trump at a campaign rally, March 16, 2024, in Vandalia, Ohio.
Sen. J.D. Vance, R-Ohio, right, points toward Republican presidential candidate former President Donald Trump at a campaign rally, March 16, 2024, in Vandalia, Ohio. Trump on July 15 announced the first-term Ohio senator as his running mate.
Jeff Dean/AP
Federal In Wake of Trump Assassination Attempt, Biden Calls for Unity and Investigation Gets Underway
President Biden condemns violence, the FBI searches for a motive, and Trump heads to RNC.
3 min read
Republican presidential candidate former President Donald Trump is surrounded by U.S. Secret Service agents at a campaign rally, Saturday, July 13, 2024, in Butler, Pa.
Former President Donald Trump is surrounded by U.S. Secret Service agents after being struck by gunfire at a campaign rally, Saturday, July 13, 2024, in Butler, Pa. The day after the attempted assasination of the Republican nominee for president, Trump arrived in Milwaukee ahead of the start of the Republican National Convention and President Joe Biden gave a prime-time address, saying "politics must never be a literal battlefied. God forbid, a killing field."
Evan Vucci/AP
Federal Project 2025 and the GOP Platform: What Each Says About K-12 in a 2nd Trump Term
A side-by-side look at what the two policy documents say on key education topics.
1 min read
Republican presidential candidate former President Donald Trump speaks at a campaign rally at Trump National Doral Miami, Tuesday, July 9, 2024, in Doral, Fla.
Former President Donald Trump speaks at a campaign rally at Trump National Doral Miami, Tuesday, July 9, 2024, in Doral, Fla.
Rebecca Blackwell/AP
Federal What the 2024 GOP Platform Says About K-12 and What It Would Mean If Trump Wins
We break down what the GOP's 2024 policy platform says about education.
7 min read
Republican presidential candidate former President Donald Trump speaks at a campaign rally at Trump National Doral Miami, Tuesday, July 9, 2024, in Doral, Fla.
Former President Donald Trump speaks at a campaign rally at Trump National Doral Miami, Tuesday, July 9, 2024, in Doral, Fla.
Rebecca Blackwell/AP