Corrected: This story originally misstated the value of the current What Works Clearinghouse contract. The story also should have made clear that while the clearinghouse gives its highest ratings to randomized-control studies, it also considers other kinds of studies that compare treatment and control groups.
The U.S. Department of Education wants its What Works Clearinghouse to become more nimble and more relevant to educators in the field over the next five years, according to bid specifications for potential contractors that the agency published last month.
The American Institutes of Research, based in Washington, currently holds a five-year, $23.4 million contract to run the project, which was begun by the department’s Institute of Education Sciences in 2002 as an online source for independent reviews of research on “what works” in education. But the AIR’s contract is due to end this year, requiring the institute to open bidding on a new one.
Read the related story,
‘What Works’ Reviewers Find No Learning Edge for Leading Math Texts
“I would call this a midcourse correction,” said James W. Kohlmoos, the president of the National Knowledge Industry Association, a Washington-based trade group, referring to the “statement of work” published online Dec. 19 in Federal Business Opportunities. “I think everybody who has been involved with this the first five years recognizes the need for some fixes, particularly when it comes to usability and relevance.”
Faster Turnaround Sought
As of last week, the clearinghouse featured 58 “intervention reports” on programs in character education, mathematics, early-childhood education, and other areas. But critics have complained that the reviews have been slow in coming and have too often yielded discouraging conclusions.
Experts disagree over whether the site’s rocky start is attributable to its meticulous research standards, a general lack of high-quality research in the field, or both. (“‘One Stop’ Research Shop Seen as Slow to Yield Views That Educators Can Use,” Sept. 27, 2006.)
Because the competition is ongoing, IES officials declined to comment on their vision for the clearinghouse.
But the new guidelines say the IES wants a contractor to devise a “streamlined, fast-response” review process for What Works reports. The statement of work also calls for producing reviews of the “best available evidence when rigorous research is lacking,” and for reporting information on the costs of implementing the programs and practices the clearinghouse reviews.
The plan describes a wide range of tasks, both new and continuing. For instance, it calls for widening reviews beyond randomized-control trials by including evidence from single-case studies, “high-quality” quasi-experiments, and other kinds of experimental designs in the mix.
The agenda also calls for two national registries—one with completed and ongoing randomized studies in education, and another listing the programs and practices schools are using.
In addition, the institute wants the clearinghouse to produce “practice guides,” for which the standards of evidence can be slightly lower, to give educators and policymakers quick, practical guidance on key educational or policy questions. The clearinghouse is now working on producing such publications, although none has been issued.
The institute also envisions having clearinghouse reviewers produce reviews of important new studies within 48 hours, and evaluate some of the other, continuing research projects that it funds.
Bidders have until Feb. 9 to show how they would fulfill the new work requirements. The AIR is expected to bid again.
