Federal

‘One Stop’ Research Shop Seen as Slow to Yield Views That Educators Can Use

By Debra Viadero — September 26, 2006 4 min read
  • Save to favorites
  • Print

Optimism was running high in September 2002 when the U.S. Department of Education unveiled its plans for a clearinghouse to vet effectiveness studies on educational programs and practices. The hope was that the What Works Clearinghouse, within just a few years, would become an online one-stop shop where practitioners and policymakers could get the lowdown on what research has to say about what works in education.

But four years and nearly $23.4 million later, critics call the What Works Clearinghouse the “nothing works” clearinghouse. As of last week, its Web site had posted reports for only 32 interventions—only eight of which have a “positive” or “potentially positive” effectiveness rating.

Education Trust Director Kati Haycock speaks last week to the National Board of Education Sciences as board member F. Philip Handy, left, and Grover J. "Russ" Whitehurst, the director of the IES, look on.

“I can’t tell if that’s because so little works, or because the clearinghouse has been so sluggish or so meticulous,” said Chester E. Finn Jr., an assistant secretary for educational research and improvement in the U.S. Department of Education during the Reagan administration. He is now the president of the Thomas B. Fordham Foundation, a Washington-based think tank.

The clearinghouse posted its first reports—10 study reviews on middle school math and peer-assisted learning strategies—in July 2004. Nearly two more years passed before another batch of products rolled off the line.

Acknowledging that the Institute of Education Science’s clearinghouse has not yet lived up to its promise, the site’s developers this month stepped up report production. Seven new reports on programs in the areas of character development, elementary school mathematics, and English-language learning were added on Sept. 14, and project directors plan to publish 40 to 50 more over the next six months.

“It’s been much harder than anybody thought,” said Grover J. “Russ” Whitehurst, the director of the IES, the Education Department’s main research arm and the agency in charge of the clearinghouse. “Now we’ve finally gotten to the point where we can get something out of the investment.”

Rules-Based Process

Still, some critics wonder if the flurry of reports is too little, too late. With growing numbers of schools failing to hit their improvement targets under the No Child Left Behind Act, it’s more important than ever to get practitioners the information they need to raise student achievement, according to analysts.

Examining the Evidence

Since its launch in 2002, the What Works Clearinghouse has reviewed research in four topical areas, including 31 studies on character education, 76 on middle school math, one on English-language learning, and 61 on elementary school math.

BRIC ARCHIVE

SOURCE: What Works Clearinghouse

“It’s gotten to a point where it’s very frustrating to the field, and they feel there are no products coming out,” said Christopher T. Cross. Currently the chairman of Cross & Joftus, an educational consulting firm based in Bethesda, Md., and Danville, Calif., Mr. Cross served in the Education Department as assistant secretary for research under President George H.W. Bush.

The research pickings at the clearinghouse are slim, observers say, because federal officials overestimated the number of studies in the field that could meet the rigorous methodological standards they set.

Even some developers of curricular programs that get comparatively favorable reviews from the clearinghouse contend the screening criteria is unrealistic.

“I find it annoying that even the best-researched mathematics curriculum gets at best left-handed compliments,” said Andrew C. Isaacs, an author of Everyday Mathematics, an elementary school program rated “potentially positive” by the clearinghouse. (“Much-Used Elementary Math Program Gets Qualified Nod From U.S. Ed. Dept.,” Sept. 20, 2006.)

Rebecca S. Herman, the project director for the clearinghouse, said it also took longer than anticipated to build an infrastructure for the systematic reviews.

“We had a pretty good sense of what the standards for review were from the beginning,” she said. “But once you get into reviewing studies and findings, you find lots of subtle variations. If you want to have a rules-based process, then you have to develop all kinds of very specific rules.”

From 2002 to 2004, Ms. Herman said, developers revamped procedures to make reports more useful. The early study reviews critiqued individual studies on particular interventions, giving them check marks, for instance, if they were deemed to “meet the standard” or “meet the standard with reservations.”

The more user-friendly “intervention reports” added this year contain program descriptions, implementation costs, and simple Consumer Reports-style ratings that summarize a program’s overall effectiveness.

The revision also added a “potentially promising” effectiveness rating to broaden practitioners’ choices.

Rival Database?

In the interim, though, other What Works-style projects are stepping into the breach left open by the paucity of scientifically proven programs.

The Education Department’s own office of planning, evaluation, and policy, for instance, is developing a database of “promising practices” that are used in schools and districts with a record of raising achievement among non-English-speaking students. Thomas W. Luce III, the outgoing assistant secretary for that office, said U.S. Secretary of Education Margaret Spellings called for the database in response to practitioners’ requests for more help.

“She said, look, you all need to start finding out what are we learning that has a good chance of working,” he said. “I don’t know that it in any way reflects negatively on IES.” The standards for inclusion in the new database, though not yet fully developed, will be admittedly lower than the screens that What Works reviewers use, Mr. Luce added.

Mr. Whitehurst, the head of the IES, declined in an interview this month to comment on the database, which is expected to be unveiled this winter. “I believe and I hope,” he said, “that the department doesn’t want to send out conflicting signals on what the standards are.”

Associate Editor David J. Hoff contributed to this report.
A version of this article appeared in the September 27, 2006 edition of Education Week as ‘One Stop’ Research Shop Seen as Slow To Yield Views That Educators Can Use


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal New Federal Team to Work on Puerto Rico School Improvement, Oversight
The Puerto Rico Education Sustainability Team will focus on creating better learning environments and improving financial management.
3 min read
Secretary of Education Miguel Cardona visits the Emilio Delgado School in Corozal on June 30, 2021 during a visit to Puerto Rico.
Secretary of Education Miguel Cardona visits the Emilio Delgado School in Corozal on June 30, 2021 during a visit to Puerto Rico.
Teresa Canino Rivera/GDA via AP
Federal Pandemic Tests Limits of Cardona's Collaborative Approach as Education Secretary
He's sought the image of a veteran educator among former peers, but COVID has forced him to take a tough stance toward some state leaders.
10 min read
Education Secretary Miguel Cardona and Schools Chancellor Meisha Porter speak to Mia Arias, 10, during their visit to P.S. 5 Port Morris, a Bronx elementary school, Tuesday, Aug. 17, 2021 in New York.
U.S. Secretary of Education Miguel Cardona and Schools Chancellor Meisha Porter speak to Mia Arias, 10, during a visit to P.S. 5 Port Morris, a Bronx elementary school, last month.
Brittainy Newman/AP
Federal White House Launches Hispanic Education Initiative Led by Miguel Cardona
President Joe Biden said his administration intends to address the "systemic causes" of educational disparities faced by Hispanic students.
2 min read
Education Secretary Miguel Cardona writes down and draws positive affirmations on poster board with students during his visit to P.S. 5 Port Morris, a Bronx elementary school, Tuesday, Aug. 17, 2021 in New York.
U.S. Secretary of Education Miguel Cardona visits students in New York City at P.S. 5 Port Morris, a Bronx elementary school in the Bronx last month.
Brittainy Newman/AP
Federal Feds Add Florida to List of States Under Investigation Over Restrictions on Mask Mandates
The Education Department told the state Sept. 10 it will probe whether its mask rule is violating the rights of students with disabilities.
3 min read
Surrounded by lawmakers, Florida Gov.Ron DeSantis speaks at the end of a legislative session on April 30, 2021, in Tallahassee, Fla.
Surrounded by lawmakers, Florida Gov.Ron DeSantis speaks at the end of a legislative session on April 30, 2021, in Tallahassee, Fla.
Wilfredo Lee/AP