Education Letter to the Editor

‘What Works’ Dialogue

September 22, 2004 2 min read
  • Save to favorites
  • Print

Clearinghouse Responds to a Critic

To the Editor:

We write in response to the letters by Michael Pressley regarding the What Works Clearinghouse (“Education’s Clearinghouse,” Letters, July 28, 2004; “‘What Works’ Divergence,” Letters, Sept. 1, 2004). They raise important issues regarding our recently released reports and overall operation, which we address below.

Mr. Pressley notes that only a limited number of studies have passed the middle-school-math-curricula review. While true, that fact will not surprise many math experts, given the small research base in this area. Further, as part of our monthly release strategy, more WWC reports in this area are being prepared even as we write.

Mr. Pressley’s concerns regarding the information on studies identified as “Does not pass screen” are well-taken. In keeping with our commitment to transparency and continuous improvement, a fuller explanation of this process will be available soon at www.whatworks.ed.gov.

Another concern he expressed regards the use of peer review in the WWC process. We differ from previous efforts at research review and reporting, which often relied on expert panels to make judgments. Instead, the WWC has built an ongoing research-review enterprise that relies on two key elements: judgments guided by a set of pre-established, rigorous decision rules developed with the help of external experts, and extensive quality control to make sure the rules are applied in a consistent manner.

Further, the WWC’s quality-control process does include peer review at several critical points. For each topic, the WWC recruits external, nationally recognized senior content advisers, or scas, and methodology consultants, or mcs. All our reports are reviewed by the original study author (first author), the WWC’s technical advisory group, and the topic SCAS and MCS. The documents that guide the WWC rating process undergo peer review. The upcoming WWC “intervention reports” will be peer-reviewed.

Perhaps Mr. Pressley’s greatest concern is the inclusion of a study in our review of peer-assisted learning, or pal, in which he participated as a secondary author (Lysynchuk, Pressley, & Vye, 1990). We debated whether this study qualified for review. In particular, we discussed the issue with our pal senior content advisers, authors of a meta-analysis of pal studies. They had wrestled with the same issue, and agreed to support whichever decision we made. Based on our desire to provide educators as much information as possible on peer-assisted learning, we reviewed the study.

Regarding this, Mr. Pressley also questions the practice of writing our own summaries of reviewed studies. We believe this is important to provide our users with a standardized, highly accessible reporting mechanism. This approach builds on the advice of educator focus groups. Following our standard practice, we contacted the study’s primary author, Linda M. Lysynchuk, for feedback on the draft WWC report. She found it to be acceptable and requested no changes.

We encourage readers to reach us directly at info@whatworks.ed.gov, for we know that their comments, like Mr. Pressley’s, will help to increase the WWC’s value to decisionmakers in education.

Becki Herman

Project Director

Bob Boruch

Principal Investigator

American Institutes for Research

Washington, D.C.


Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: October 11, 2023
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: September 27, 2023
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Briefly Stated: September 20, 2023
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education From Our Research Center What's on the Minds of Educators, in Charts
Politics, gender equity, and technology—how teachers and administrators say these issues are affecting the field.
1 min read
Stylized illustration of a pie chart
Traci Daberko for Education Week