To the Editor:
Thank you for your well-researched article on the federal Institute of Education Sciences’ What Works Clearinghouse (“‘One Stop’ Research Shop Seen as Slow to Yield Views That Educators Can Use,” Sept. 27, 2006). We would like to clarify a few points, however.
The What Works Clearinghouse is keenly aware of the urgent need for reports on findings. But because the clearinghouse needs to combine rigorous reviews with easy-to-read and useful reports, we invested more deeply in development than was originally planned. The resulting solid, scientific infrastructure is now paying off, as the clearinghouse will release approximately 70 reports in the next seven months.
Rather than relying on expert-consensus judgment, the clearinghouse conducts “rules based” reviews. All decisions are based on the clearinghouse’s standards and technical guidelines. The standards were established early, but extensive review by experts, drawing from hundreds of studies, identified nearly 70 major technical issues that required painstaking development of more explicit guidance. (Additional information on this is available at http://whatworks.ed.gov/reviewprocess/workpapers.html.) Despite the additional time, the guidance facilitates quicker reviews and helps researchers think through their study designs—a significant contribution to the field.
Moreover, the What Works Clearinghouse refocused from study to intervention reports, redesigned the intervention reports, and developed a rating system to characterize interventions’ effects, to achieve the balance between technical accuracy and accessibility. It developed a system for translating varied study findings on an intervention into scientifically valid conclusions and digestible ratings.
The clearinghouse has released reports weekly for the past month. We will continue these regular releases, and are able to do so thanks to the work done early on. While we would have preferred to deliver more reports earlier, it was worthwhile to take the time to do it right. Educators don’t just need a quick fix; they need accurate and accessible information to make well-informed decisions. We all need to have confidence that when we refer to “the evidence,” we are drawing from the most accurate findings available.
Rebecca Herman
Project Director
What Works Clearinghouse
Washington, D.C.
