School & District Management

Panel Defines ‘Science’ of Education Research

By Debra Viadero — December 12, 2001 4 min read

When it comes to guiding principles, research in education is not all that different from research in the natural or social sciences, a panel of prominent researchers concludes in a new report.

Read “Scientific Inquiry in Education,” produced by the National Research Council and published by the National Academy Press.

Produced by a committee of the National Research Council, the report was put together partly in response to debates taking place among researchers and policymakers on the meaning of “scientifically based” research in education, a field often criticized for the uneven quality of its studies.

Some experts argue that the “gold standard” for education research should be strictly controlled, random-assignment studies. Others suggest that the methods of inquiry in the field can rightly be considerably broader, encompassing everything from classical experiments to well-written novels.

In its report, which was released last week, the 16-member panel favors neither the strictest nor the broadest position. Rather, it argues that studies in education, like studies in biology, physics, anthropology, and other areas, should pose significant questions, link to relevant theory, and use research tools that are appropriate for answering those questions.

Researchers should also systematically rule out counter-explanations, replicate their findings, and put their work and their data out to be scrutinized by their colleagues and the public, the report says.

“It’s not overly shocking, but it certainly sends the signal that anything doesn’t go,” Richard J. Shavelson, the Stanford University education professor who chaired the panel, said of the stance the group takes.

Sorting Out Science

While education researchers should do more randomized experiments, Mr. Shavelson said, the methodology might not be appropriate, for example, for studying the impact of teacher salaries on student achievement or for tracing young children’s language development.

At the same time, the report notes, some popular forms of education studies, such as “portraiture,” a method in which the researcher and subject collaborate in writing literary-style portraits, are not science.

Such work, said Robert F. Boruch, a University of Pennsylvania professor of education and statistics who served on the panel, “is a way of enlarging the way we think about phenomena in the same way an artist enlarges the way we think about color and perspective, but it’s not particularly science.”

The panel’s report is as notable for its diverse membership as it is for its findings. In addition to Mr. Boruch, who champions randomized experiments, the group includes an anthropologist, a cell biologist, an education historian, an economist, a reading-disabilities researcher, a chemistry professor, and a testing expert.

Convened a year ago, the panel met monthly in order to finish before Congress took up legislation to reauthorize the primary federal research office for education, the Department of Education’s office of educational research and improvement. As part of that process, some federal lawmakers last year took at stab at writing into law a definition of scientific education research—an attempt that many education researchers criticized as being too restrictive.

“We certainly hope that our report makes it unnecessary for federal legislators to say what they think science research is,” Mr. Shavelson said.

The report is also timely because President Bush and members of his administration have expressed a keen interest in promoting “scientifically based research” in education.

To encourage the kind of “scientific culture” in the OERI that might foster rigorous research, the report recommends insulating the agency’s research from political interference. It stops short, however, of describing how that might happen.

“You can’t do good research if people in Congress or in the executive branch are constantly coming and telling you what to do,” said panel member Ellen Condliffe Lagemann. She is president of the Chicago-based Spencer Foundation, which promotes education research, including coverage of the subject in Education Week.

Echoing recent reports on education research, the report says that federal support for education research is weak— and getting weaker. While total funding for the research agency grew tenfold from 1980 to 2000, the percentage of those dollars supporting studies fell sharply in the 1980s and now hovers around 15 percent.

The panelists also called for recruiting more experienced researchers to staff the agency and to review grant proposals.

Kenji Hakuta, the Stanford University education professor who chairs the National Educational Research Policy and Priorities Board, called the study good advice. His board, which advises the Education Department on its research operations, commissioned the $600,000 study.

“It says first and foremost the agency has to be populated by people who understand what the culture of science consists of,” he said, “and that the review of proposals needs to be managed through a peer-review process that reflects that culture.”

Department officials declined to comment on the report.

Related Tags:

A version of this article appeared in the December 12, 2001 edition of Education Week as Panel Defines ‘Science’ of Education Research

Events

Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How to Power Your Curriculum With Digital Books
Register for this can’t miss session looking at best practices for utilizing digital books to support their curriculum.
Content provided by OverDrive
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Embracing Student Engagement: The Pathway to Post-Pandemic Learning
As schools emerge from remote learning, educators are understandably worried about content and skills that students would otherwise have learned under normal circumstances. This raises the very real possibility that children will face endless hours
Content provided by Newsela

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion Are Your Leadership Practices Good Enough for Racial Justice?
Scratch being a hero. Instead, build trust and reach beyond school walls, write Jennifer Cheatham and John B. Diamond.
Jennifer Cheatham & John B. Diamond
5 min read
Illustration of leadership.
Collage by Laura Baker/Education Week (Images: DigitalVision Vectors, iStock, Getty)
School & District Management We Pay Superintendents Big Bucks and Expect Them to Succeed. But We Hardly Know Them
National data is skimpy, making it hard to know what influences superintendents' decisions to move on, retire, or how long they stay. Why?
8 min read
Conceptual image of tracking with data.
marcoventuriniautieri/iStock/Getty
School & District Management Data For the First Time in the Pandemic, a Majority of 4th Graders Learn in Person Full Time
The latest monthly federal data still show big racial and socioeconomic differences in who has access to full-time in-person instruction.
3 min read
Student with backpack.
surasaki/iStock/Getty
School & District Management From Our Research Center To Offer Remote Learning in the Fall or Not? Schools Are Split
An EdWeek Research Center survey shows that nearly 4 of every 10 educators say their schools will not offer any remote instruction options.
4 min read
Image of a teacher working with a student through a screen session.
Ridofranz/iStock/Getty