School & District Management

Report Urges Use of Medical-Style Research in Education

By Debra Viadero — November 27, 2002 6 min read
  • Save to favorites
  • Print

The federal government can meet mandates to foster the use of “scientifically based research” in education by aggressively promoting randomized, controlled trials, a report released last week argues.

Commonly used in medicine and pharmacology, such trials entail the random assignment of subjects to either an experimental group or a comparison group. Such investigations are less common—and sometimes controversial—in education and some other social-science fields.

The report by the Coalition for Evidence-Based Policy says that the federal government should make a concerted effort to sponsor more such studies in education and to prod states, schools, and districts to use the resulting findings. The coalition is sponsored by the Council for Excellence in Government, a nonprofit, better-government group based here.

The report, “Bringing Evidence-Driven Progress to Education,” is available from the Coalition for Evidence-Based Policy. (Requires Adobe’s Acrobat Reader.)

“There’s been no improvement in education over the last 30 years, despite a 90 percent increase in real public spending per pupil,” Jon Baron, the coalition’s executive director, contended. “What our coalition proposes in order to spark progress is a major shift in public policy toward funding and effectively using randomized trials.”

The 15-member panel that produced the report includes David Kessler, a former commissioner of the Food and Drug Administration; the education historian Diane Ravitch, who served as the Department of Education’s assistant secretary for research during the first Bush administration; the Nobel Prize-winning economist Robert Solow; Robert Boruch, a University of Pennsylvania education and statistics professor; and Robert Slavin, the Johns Hopkins University researcher who co- founded the Success for All program.

Though just a year old, the coalition already has attracted the attention of top federal education officials, several of whom attended a policy forum here last week that centered on the report’s recommendations. Financed by the Washington-based William T. Grant Foundation, the report, in fact, was the product of a collaboration between federal education officials and the coalition.

“We’re reviewing these recommendations right now with great interest, and we’re eager to learn more from them,” Secretary of Education Rod Paige told the group.

Rare Spotlight

For a topic as arcane as research methodology, such attention is rare. The subject has risen on the federal agenda, however, with the recent passage of two major pieces of education legislation: the “No Child Left Behind” Act of 2001 and the Education Sciences Reform Act of 2002.

Using similar language, the laws either require schools and states to use only programs developed with “scientifically based” research or require the Education Department to fund only such studies.

While the legislative definitions of “scientifically based” or “scientifically valid” research clearly favor randomly controlled studies, they also allow for other kinds of research methodologies, such as quasi-experimental studies.

Researchers in many fields view randomized, controlled studies as the “gold standard’’ for research on the effectiveness of particular approaches. But in education, some scholars caution that such studies can be expensive and might be inappropriate for answering some types of basic research questions.

Ethical issues also arise in randomized trials when researchers have to decide whether to withhold a promising teaching strategy from one group of children while using it with another.

Gerald R. Sroufe, the government-relations director for the American Educational Research Association, said the new report may “oversell” the role that randomized, controlled trials can play in improving schools.

“It’s important to remember that experimental methodology is not science,” he added. “It’s a tool of science.”

Mr. Baron, for his part, said his group had no intention of excluding other kinds of studies from getting federal support.

“A lot of other kinds of work is needed to know where you’re going to invest your money,” he said. But ultimately, he added, any strategy or program that the federal government hopes to endorse for large- scale use should be tested in randomized trials.

Recommendations

To foster more experimental work, the coalition recommends that the Education Department:

  • Set aside more of its discretionary money for randomized trials.
  • Dedicate a small percentage—perhaps 2 percent—of the money that states now get through the No Child Left Behind Act to paying for experiments. In fiscal 2002, that would amount to $400 million.
  • In other grant programs, give grantees and applicants incentives to use results from randomized, controlled studies and to set aside their own discretionary funds to pay for more randomized experiments.

“What we learned in medicine was that the things we think we know, we don’t actually know until we subject them to randomized control trials,” Steven Goodman, an associate professor of oncology and biostatistics at the Johns Hopkins University school of medicine, told the forum.

In the 1940s, he noted, doctors held the widespread belief—based on evidence from non- randomized Australian studies—that premature infants could benefit from oxygen treatments. Medical professionals were so convinced of the efficacy of the practice, he said, that during randomized field trials of the practice, nurses would come in at night and turn up the oxygen in the babies’ cribs.

That 1954 experiment, however, showed that disproportionately high percentages of infants in the high-oxygen units were becoming blind as a result of the treatments. The practice was eventually discontinued.

“What looks unethical today can look lifesaving tomorrow,” Mr. Goodman said.

Assistant Secretary of Education Grover J. “Russ” Whitehurst, who heads the Education Department’s office of educational research and improvement, said the department has already begun channeling discretionary funds to pay for randomized experiments involving reading, community learning centers, the federal Even Start program, and 11 other areas.

The pace at which the federal government appears to be embracing such experiments alarmed some participants, who worry that its focus on methodology was too narrow.

“This is a bulldozer plowing through matchstick houses and, hopefully, what they build out of the ashes will be wonderful communities,” said Cathy Roller, the director of research and policy for the International Reading Association.

Mr. Slavin of Johns Hopkins noted, though, that the Education Department’s efforts are not unprecedented. The Comprehensive School Reform Demonstration Program, created by Congress in 1997, required states to either choose reform strategies from a federally approved list of proven programs or select another program with research evidence to back it up.

However, Mr. Slavin said, only a quarter of those grants have gone to programs, such as his, that have some experimental evidence attesting to their effectiveness.

“As time has gone on, the Department of Education has had this kind of language appear more often,” he said, referring to mandates for such evidence. “I think it’s very much yet to be seen whether it’s going to change anything.”

Related Tags:

A version of this article appeared in the November 27, 2002 edition of Education Week as Report Urges Use of Medical-Style Research in Education

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
Future of the First Amendment:Exploring Trends in High School Students’ Views of Free Speech
Learn how educators are navigating student free speech issues and addressing controversial topics like gender and race in the classroom.
Content provided by The John S. and James L. Knight Foundation
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Start Strong With Solid SEL Implementation: Success Strategies for the New School Year
Join Satchel Pulse to learn why implementing a solid SEL program at the beginning of the year will deliver maximum impact to your students.
Content provided by Satchel Pulse
Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Hour by Busy Hour: What a Principal's Day Actually Looks Like
From the time they wake up until they set the alarm at night, school leaders juggle the routine, the unexpected, and the downright bizarre.
Left, Principal Michael C. Brown talks on a radio at Winters Mill High School in Westminster, Md., on May 17, 2022. Right, Boone Elementary School principal Manuela Haberer directs students and parents in the pick-up line at the conclusion of the school day on May 19, 2022 in San Antonio, Texas.
Left, Principal Michael C. Brown talks on a radio at Winters Mill High School in Westminster, Md., on May 17, 2022. Right, Boone Elementary School principal Manuela Haberer directs students and parents in the pick-up line at the conclusion of the school day on May 19, 2022 in San Antonio, Texas.
From left, Steve Ruark and Lisa Krantz for Education Week
School & District Management Photos What School Leadership Looks Like: A Day in the Life of a Principal
A look at a typical day for one elementary school principal in Texas and a high school principal in Maryland.
1 min read
Principal Michael C. Brown, from left, talks to seniors Brady D’Anthony, 18, and Sydney Dryden, 17, at Winters Mill High School in Westminster, Md., Tuesday, May 17, 2022.
Principal Michael C. Brown, from left, talks to seniors Brady D’Anthony, 18, and Sydney Dryden, 17, at Winters Mill High School in Westminster, Md., Tuesday, May 17, 2022.
Steve Ruark for Education Week
School & District Management Schools Can Access Tons of Money for Electric Buses. Will They Use It?
Electric buses are growing more appealing as fuel prices rise, but some districts remain wary of the cost and logistics.
5 min read
Stockton Unified School District's new electric bus fleet reduces over 120,000 pounds of carbon emissions and leverages The Mobility House's smart charging and energy management system.
The new electric bus fleet at California's Stockton Unified School District is projected to reduce the district's carbon emissions.
Business Wire via AP
School & District Management Explainer Who Are Paraprofessionals and What Do They Do?
Paraprofessionals are a vital part of the classroom team with a wide range of responsibilities and skills.
1 min read