School & District Management

Panel Defines ‘Science’ of Education Research

By Debra Viadero — December 12, 2001 4 min read
  • Save to favorites
  • Print

When it comes to guiding principles, research in education is not all that different from research in the natural or social sciences, a panel of prominent researchers concludes in a new report.

Read “Scientific Inquiry in Education,” produced by the National Research Council and published by the National Academy Press.

Produced by a committee of the National Research Council, the report was put together partly in response to debates taking place among researchers and policymakers on the meaning of “scientifically based” research in education, a field often criticized for the uneven quality of its studies.

Some experts argue that the “gold standard” for education research should be strictly controlled, random-assignment studies. Others suggest that the methods of inquiry in the field can rightly be considerably broader, encompassing everything from classical experiments to well-written novels.

In its report, which was released last week, the 16-member panel favors neither the strictest nor the broadest position. Rather, it argues that studies in education, like studies in biology, physics, anthropology, and other areas, should pose significant questions, link to relevant theory, and use research tools that are appropriate for answering those questions.

Researchers should also systematically rule out counter-explanations, replicate their findings, and put their work and their data out to be scrutinized by their colleagues and the public, the report says.

“It’s not overly shocking, but it certainly sends the signal that anything doesn’t go,” Richard J. Shavelson, the Stanford University education professor who chaired the panel, said of the stance the group takes.

Sorting Out Science

While education researchers should do more randomized experiments, Mr. Shavelson said, the methodology might not be appropriate, for example, for studying the impact of teacher salaries on student achievement or for tracing young children’s language development.

At the same time, the report notes, some popular forms of education studies, such as “portraiture,” a method in which the researcher and subject collaborate in writing literary-style portraits, are not science.

Such work, said Robert F. Boruch, a University of Pennsylvania professor of education and statistics who served on the panel, “is a way of enlarging the way we think about phenomena in the same way an artist enlarges the way we think about color and perspective, but it’s not particularly science.”

The panel’s report is as notable for its diverse membership as it is for its findings. In addition to Mr. Boruch, who champions randomized experiments, the group includes an anthropologist, a cell biologist, an education historian, an economist, a reading-disabilities researcher, a chemistry professor, and a testing expert.

Convened a year ago, the panel met monthly in order to finish before Congress took up legislation to reauthorize the primary federal research office for education, the Department of Education’s office of educational research and improvement. As part of that process, some federal lawmakers last year took at stab at writing into law a definition of scientific education research—an attempt that many education researchers criticized as being too restrictive.

“We certainly hope that our report makes it unnecessary for federal legislators to say what they think science research is,” Mr. Shavelson said.

The report is also timely because President Bush and members of his administration have expressed a keen interest in promoting “scientifically based research” in education.

To encourage the kind of “scientific culture” in the OERI that might foster rigorous research, the report recommends insulating the agency’s research from political interference. It stops short, however, of describing how that might happen.

“You can’t do good research if people in Congress or in the executive branch are constantly coming and telling you what to do,” said panel member Ellen Condliffe Lagemann. She is president of the Chicago-based Spencer Foundation, which promotes education research, including coverage of the subject in Education Week.

Echoing recent reports on education research, the report says that federal support for education research is weak— and getting weaker. While total funding for the research agency grew tenfold from 1980 to 2000, the percentage of those dollars supporting studies fell sharply in the 1980s and now hovers around 15 percent.

The panelists also called for recruiting more experienced researchers to staff the agency and to review grant proposals.

Kenji Hakuta, the Stanford University education professor who chairs the National Educational Research Policy and Priorities Board, called the study good advice. His board, which advises the Education Department on its research operations, commissioned the $600,000 study.

“It says first and foremost the agency has to be populated by people who understand what the culture of science consists of,” he said, “and that the review of proposals needs to be managed through a peer-review process that reflects that culture.”

Department officials declined to comment on the report.

Related Tags:

A version of this article appeared in the December 12, 2001 edition of Education Week as Panel Defines ‘Science’ of Education Research

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
The Future of the Science of Reading
Join us for a discussion on the future of the Science of Reading and how to support every student’s path to literacy.
Content provided by HMH
Mathematics K-12 Essentials Forum Helping Students Succeed in Math
Student Well-Being Live Online Discussion A Seat at the Table: The Power of Emotion Regulation to Drive K-12 Academic Performance and Wellbeing
Wish you could handle emotions better? Learn practical strategies with researcher Marc Brackett and host Peter DeWitt.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion The Stunning Resignation of UVA President Jim Ryan—and Why It Matters
The university president’s departure is more than just a headline. It’s a lesson in leadership.
2 min read
Opinion Licensed Not for Reuse Wait What FCG
Canva
School & District Management In Their Own Words This Custodian Got Students to Stop Vandalizing and Take Pride in Their School
Andy Markus, the 2025 Education Support Professional of the Year, helped boost behavior and engagement in his Utah district.
5 min read
Andy Markus, the head custodian at Draper Park Middle School, in Draper, Utah, sits for a portrait during the National Education Association's 2025 Representative Assembly in Portland, Ore., on July 3, 2025. Markus was named the 2025 NEA Education Support Professional (ESP) of the Year.
Andy Markus, the head custodian at Draper Park Middle School, in Draper, Utah, sits for a portrait during the National Education Association's 2025 representative assembly in Portland, Ore., on July 3, 2025. Markus was named the 2025 NEA Education Support Professional of the Year for his mentorship of students.
Kaylee Domzalski/Education Week
School & District Management What the Research Says About School Boards: How Much Conflict Really Is There?
Plus, how competitive are board elections? How much do teachers' union endorsements matter?
7 min read
Houston ISD's appointed school board votes on the "District of Innovation" status during their monthly work session meeting at HISD Central Office on Sept. 7, 2023 in Houston.
Houston's appointed school board takes a vote during a meeting on Sept. 7, 2023 in the district's central office. A number of studies from recent years have answered questions about school boards' makeup, how competitive board elections are, whether conflict is on the rise, and more.
Karen Warren/Houston Chronicle via AP
School & District Management Opinion How a Weekly Email to My Staff Made Me a Better District Leader
Writing helps make sense out of what feels messy and focus us on what's most important.
George Philhower
5 min read
Blue hand holding red pen.
DigitalVision Vectors/Getty + Education Week