School & District Management

Review Process for U.S. Education Research Approved

By Debra Viadero — January 31, 2006 3 min read
  • Save to favorites
  • Print

A national research-advisory board last week approved guidelines to govern the outside review processes that the Department of Education uses to screen and approve the studies and reports that it funds.

Taking a cue from federal science agencies, such as the National Institutes of Health and the National Science Agency, the guidelines call for standing panels of reviewers to screen many of the research proposals funded by the Education Department’s primary research arm, the Institute of Education Sciences.

They also formalize a special office within the institute that functions almost like the editor of a professional, peer-reviewed journal and coordinates reviews for most of the reports published with the Education Department’s imprimatur.

“This is a scientific review process that is truly independent of staff,” said advisory- board member Jon Baron, the executive director of the Washington-based Council for Excellence in Government. “It goes a long way toward establishing the institute’s credibility as an independent, nonideological source of authority on education research.”

Critics have long said the Education Department’s peer-review procedures are partly to blame for the low quality of federally funded education research. A department-commissioned report in 1999, for instance, said that such procedures were sometimes haphazard and that reviewers often lacked expertise in research methodology.

When the institute was created in 2002, federal officials began to set in place more-structured procedures for handling outside reviews. But, under federal law, those procedures have to be formally approved by the National Board of Education Sciences, a 14-member board whose members are nominated by the president. At its meeting last week, the board essentially approved most of the mechanisms already in place, adding some small changes of its own.

“What’s new here is the process of making these procedures transparent and available to the public,” said Gerald E. Sroufe, the director of government relations for the American Educational Research Association, a Washington-based group that represents 22,000 researchers.

Two Review Tracks

The guidelines approved on Jan. 24 set up procedures for two kinds of research-review processes—one for deciding which research proposals the department will fund, and one for signing off on reports on federally financed education studies.

Spanning about a year, the process for research proposals begins in March, when the department announces its research competitions. Panels meet to screen proposals twice a year, once in October or November and again in February or March.

Proposals are reviewed by either a standing peer-review panel or a single-session panel. Typically composed of 20 reviewers, the standing panels vet proposals for all the studies that fall under a particular topic area, such as reading or mathematics education. Serving anywhere from one to three years, most of the members of those panels have to be approved by the institute’s director, Grover J. “Russ” Whitehurst.

“Basically, we’re looking for highly qualified experts who are at least as good as the [principal investigators] who are applying,” said Mr. Whitehurst.

However, one panel member questioned whether reviewers should include experts in the content area as well as researchers.

“If the report or the proposal is about math, then there should be mathematicians,” said R. James Milgram, a mathematics professor at Stanford University. He said he and other professional mathematicians, in reviews of state and federal math assessments, have found mathematical errors in 20 percent to 25 percent of the questions.

But board members put off that issue until the panel meets again in May.

At members’ urging, the guidelines were amended to specify a process for “inviting” reviewers to disclose any methodological or ideological biases they have at the point when the review panels convene—a policy that other federal science agencies follow.

For reports produced by the institute’s four research centers, the guidelines call for two stages of review. The first stage occurs at the research centers themselves; the second takes place in the institute’s office of standards and review, under the newly approved procedures. Officials in that office either have the reports reviewed by staff or recruit outside experts—two or more for each report—to critique them.

Related Tags:

Events

School Climate & Safety K-12 Essentials Forum Strengthen Students’ Connections to School
Join this free event to learn how schools are creating the space for students to form strong bonds with each other and trusted adults.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Creating Confident Readers: Why Differentiated Instruction is Equitable Instruction
Join us as we break down how differentiated instruction can advance your school’s literacy and equity goals.
Content provided by Lexia Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
Future-Proofing Your School's Tech Ecosystem: Strategies for Asset Tracking, Sustainability, and Budget Optimization
Gain actionable insights into effective asset management, budget optimization, and sustainable IT practices.
Content provided by Follett Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Schools Successfully Fighting Chronic Absenteeism Have This in Common
A White House summit homed in on chronic absenteeism and strategies to reduce it.
6 min read
An empty elementary school classroom is seen on Aug. 17, 2021 in the Bronx borough of New York. Nationwide, students have been absent at record rates since schools reopened after COVID-forced closures. More than a quarter of students missed at least 10% of the 2021-22 school year.
An empty elementary school classroom is seen on Aug. 17, 2021 in the Bronx borough of New York. A White House summit on May 15, 2024, brought attention to elevated chronic absenteeism and strategies districts have used to fight it.
Brittainy Newman/AP
School & District Management From Our Research Center Here's What Superintendents Think They Should Be Paid
A new survey asks school district leaders whether they're paid fairly.
3 min read
Illustration of a ladder on a blue background reaching the shape of a puzzle piece peeled back and revealing a Benjamin Franklin bank note behind it.
iStock/Getty
School & District Management Q&A How K-12 Leaders Can Better Manage Divisive Curriculum and Culture War Debates
The leader of an effort to equip K-12 leaders with conflict resolution skills urges relationship-building—and knowing when to disengage.
7 min read
Katy Anthes, Commissioner of Education in Colorado from 2016- 2023, participates in a breakout session during the Education Week Leadership Symposium on May 3, 2024.
Katy Anthes, who served as commissioner of education in Colorado from 2016-2023, participates in a breakout session during the Education Week Leadership Symposium on May 3, 2024. Anthes specializes in helping school district leaders successfully manage politically charged conflicts.
Chris Ferenzi for Education Week
School & District Management Virginia School Board Restores Confederate Names to 2 Schools
The vote reverses a decision made in 2020 as dozens of schools nationwide dropped Confederate figures from their names.
2 min read
A statue of confederate general Stonewall Jackson is removed on July 1, 2020, in Richmond, Va. Shenandoah County, Virginia's school board voted 5-1 early Friday, May 10, 2024, to rename Mountain View High School as Stonewall Jackson High School and Honey Run Elementary as Ashby Lee Elementary four years after the names had been removed.
A statue of confederate general Stonewall Jackson is removed on July 1, 2020, in Richmond, Va. Shenandoah County, Virginia's school board voted 5-1 early Friday, May 10, 2024, to rename Mountain View High School as Stonewall Jackson High School and Honey Run Elementary as Ashby Lee Elementary four years after the names had been removed.
Steve Helber/AP