Federal

Federal Reading Review Overlooks Popular Texts

By Kathleen Kennedy Manzo — August 28, 2007 4 min read
  • Save to favorites
  • Print

When a federal review of beginning-reading programs was commissioned four years ago, experts and educators hoped it would help school leaders sift through the vast marketplace of instructional materials and find those most effective for improving achievement. But the long-awaited study by the What Works Clearinghouse, released this month, may not fully deliver on that promise.

In fact, the analysis found that few comprehensive or supplemental programs have empirical proof that they work. And none of the most popular commercial programs on the market—including McGraw-Hill’s Open Court, Scott Foresman Reading, and Houghton Mifflin Reading, which have earned hundreds of millions of dollars in sales to districts—had sufficiently rigorous studies to be included in the review by the clearinghouse.

“They tended not to have studies with randomized-control trials or with experimental designs that met the clearinghouse’s evidence standards,” said Jill Constantine, the principal director of the review.

Most of the programs deemed to have “positive effects” or “potentially positive effects” in the review were supplemental or intervention programs, not core reading series. In the review of nearly 900 studies, just 51 met the standard for evidence, meaning just one or two for each program.

More than 120 other programs, however, had no studies that met the requirements.

The complexity of the study, and the scarcity of suitable research on the topic, point to the difficulty of determining what works in the classroom, said James W. Kohlmoos, the president of the Knowledge Alliance, a Washington-based nonprofit organization that works with education groups to translate research findings to instruction.

“In the very early stages, we thought it was something that could be of value, but it’s only of marginal value right now,” Mr. Kohlmoos said of the federal review. “We all wish there was more in there and the results were more definitive,” he said, “but it just shows that this is a far more complex and difficult effort than we all originally thought.”

Smaller Programs Reviewed

The clearinghouse, which the U.S. Department of Education’s Institute of Education Sciences set up in 2002 to vet research on “what works” in education, has given few of its coveted positive ratings.

What Works: Effectiveness of Reading Programs

Source: Institute of Education Sciences

BRIC ARCHIVE

Just one of the reading programs examined was found to have positive effects or potentially positive effects across all four of the domains in the review: alphabetics, fluency, comprehension, and general reading achievement. That program, Reading Recovery, involving intensive, one-on-one tutoring, has drawn criticism over the past few years from prominent researchers and federal officials who claim it is costly and lacks evidence that it helps struggling readers.

Success for All, a whole-school-reform program developed at Johns Hopkins University in Baltimore, got the favorable rating on alphabetics and general reading achievement, but mixed results on comprehension. Voyager Universal Literacy System, published by the Dallas-based Voyager Learning, was found to have potentially positive effects on alphabetics but potentially negative effects on comprehension. Accelerated Reader, distributed by Renaissance Learning Inc., was found to have a potentially positive impact on comprehension and general reading achievement.

The Success for All review is not as glowing as previous federal and private studies, which have put it in an exclusive group of programs that meet the highest standard of evidence that they are effective at teaching children to read. (“Long-Awaited Study Shows ‘Success for All’ Gains,” May 11, 2005.)

Robert E. Slavin, the founder of Success for All, said the clearinghouse review discounted such studies, which were based on large numbers of students exposed to the whole-school-reform program, and instead favored research on targeted interventions for small groups of students.

Those products—such as Start Making a Reader Today, Kaplan SpellRead, and Peer-Assisted Learning Strategies, or PALS—also got positive reviews.

But Mr. Slavin said the review is missing important information on the large commercial programs.

“The What Works Clearinghouse was designed to ... tell educators in a fair, nonpolitical, scientifically justifiable way what is the strength of evidence of various programs,” he said. “How could it not have done so on the largest programs? To go for five years without getting that job done boggles the imagination.”

In fact, the clearinghouse report lists 36 products, including some widely used series, that are in the process of being reviewed, but notes that each has just one study that meets the criteria “with reservations.”

First Appraisal

Tom Stanton, a spokesman for the New York City-based McGraw-Hill, wrote in an e-mail that the company’s programs have been developed over decades of “intense study, scientific research, and field testing.”

“The key question,” he added, “is, ‘do our programs drive student achievement?’ Our answer is that they do, and that we have results to prove it.”

The reading review, which has been under way for more than three years, is the first in an ongoing appraisal of reading programs, according to Phoebe H. Cottingham, the commissioner of the National Center for Education Evaluation and Regional Assistance, which oversees the clearinghouse.

Related Tags:

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Biden Calls for Teacher Pay Raises, Expanded Pre-K in State of the Union
President Joe Biden highlighted a number of his education priorities in a high-stakes speech as he seeks a second term.
5 min read
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
Shawn Thew/Pool via AP
Federal Low-Performing Schools Are Left to Languish by Districts and States, Watchdog Finds
Fewer than half of district plans for improving struggling schools meet bare minimum requirements.
11 min read
A group of silhouettes looks across a grid with a public school on the other side.
Vanessa Solis/Education Week via Canva
Federal Biden Admin. Says New K-12 Agenda Tackles Absenteeism, Tutoring, Extended Learning
The White House unveiled a set of K-12 priorities at the start of an election year.
4 min read
U.S. Education Secretary Miguel Cardona participates in a roundtable discussion with students from Dartmouth College on Jan. 10, 2024, on the school's campus, in Hanover, N.H.
U.S. Education Secretary Miguel Cardona participates in a roundtable discussion with students from Dartmouth College on Jan. 10, 2024, on the school's campus, in Hanover, N.H.
Steven Senne/AP
Federal Lawmakers Want to Reauthorize a Major Education Research Law. What Stands in the Way?
Lawmakers have tried and failed to reauthorize the Education Sciences Reform Act over the past nearly two decades.
7 min read
Sen. Bill Cassidy, R-La., left, joins Senate Health, Education, Labor and Pensions Committee Chairman Bernie Sanders, I-Vt., left, as Starbucks founder Howard Schultz answers questions about the company's actions during an ongoing employee unionizing campaign, at the Capitol in Washington, on March 29, 2023.
Sen. Bill Cassidy, R-La., left, joins Senate Health, Education, Labor and Pensions Committee Chairman Bernie Sanders, I-Vt., left, at the Capitol in Washington, on March 29, 2023. The two lawmakers sponsored a bill to reauthorize the Education Sciences Reform Act.
J. Scott Applewhite/AP