School & District Management

Panel Defines ‘Science’ of Education Research

By Debra Viadero — December 12, 2001 4 min read
  • Save to favorites
  • Print

When it comes to guiding principles, research in education is not all that different from research in the natural or social sciences, a panel of prominent researchers concludes in a new report.

Read “Scientific Inquiry in Education,” produced by the National Research Council and published by the National Academy Press.

Produced by a committee of the National Research Council, the report was put together partly in response to debates taking place among researchers and policymakers on the meaning of “scientifically based” research in education, a field often criticized for the uneven quality of its studies.

Some experts argue that the “gold standard” for education research should be strictly controlled, random-assignment studies. Others suggest that the methods of inquiry in the field can rightly be considerably broader, encompassing everything from classical experiments to well-written novels.

In its report, which was released last week, the 16-member panel favors neither the strictest nor the broadest position. Rather, it argues that studies in education, like studies in biology, physics, anthropology, and other areas, should pose significant questions, link to relevant theory, and use research tools that are appropriate for answering those questions.

Researchers should also systematically rule out counter-explanations, replicate their findings, and put their work and their data out to be scrutinized by their colleagues and the public, the report says.

“It’s not overly shocking, but it certainly sends the signal that anything doesn’t go,” Richard J. Shavelson, the Stanford University education professor who chaired the panel, said of the stance the group takes.

Sorting Out Science

While education researchers should do more randomized experiments, Mr. Shavelson said, the methodology might not be appropriate, for example, for studying the impact of teacher salaries on student achievement or for tracing young children’s language development.

At the same time, the report notes, some popular forms of education studies, such as “portraiture,” a method in which the researcher and subject collaborate in writing literary-style portraits, are not science.

Such work, said Robert F. Boruch, a University of Pennsylvania professor of education and statistics who served on the panel, “is a way of enlarging the way we think about phenomena in the same way an artist enlarges the way we think about color and perspective, but it’s not particularly science.”

The panel’s report is as notable for its diverse membership as it is for its findings. In addition to Mr. Boruch, who champions randomized experiments, the group includes an anthropologist, a cell biologist, an education historian, an economist, a reading-disabilities researcher, a chemistry professor, and a testing expert.

Convened a year ago, the panel met monthly in order to finish before Congress took up legislation to reauthorize the primary federal research office for education, the Department of Education’s office of educational research and improvement. As part of that process, some federal lawmakers last year took at stab at writing into law a definition of scientific education research—an attempt that many education researchers criticized as being too restrictive.

“We certainly hope that our report makes it unnecessary for federal legislators to say what they think science research is,” Mr. Shavelson said.

The report is also timely because President Bush and members of his administration have expressed a keen interest in promoting “scientifically based research” in education.

To encourage the kind of “scientific culture” in the OERI that might foster rigorous research, the report recommends insulating the agency’s research from political interference. It stops short, however, of describing how that might happen.

“You can’t do good research if people in Congress or in the executive branch are constantly coming and telling you what to do,” said panel member Ellen Condliffe Lagemann. She is president of the Chicago-based Spencer Foundation, which promotes education research, including coverage of the subject in Education Week.

Echoing recent reports on education research, the report says that federal support for education research is weak— and getting weaker. While total funding for the research agency grew tenfold from 1980 to 2000, the percentage of those dollars supporting studies fell sharply in the 1980s and now hovers around 15 percent.

The panelists also called for recruiting more experienced researchers to staff the agency and to review grant proposals.

Kenji Hakuta, the Stanford University education professor who chairs the National Educational Research Policy and Priorities Board, called the study good advice. His board, which advises the Education Department on its research operations, commissioned the $600,000 study.

“It says first and foremost the agency has to be populated by people who understand what the culture of science consists of,” he said, “and that the review of proposals needs to be managed through a peer-review process that reflects that culture.”

Department officials declined to comment on the report.

Related Tags:

A version of this article appeared in the December 12, 2001 edition of Education Week as Panel Defines ‘Science’ of Education Research

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Data Webinar
Education Insights with Actionable Data to Create More Personalized Engagement
The world has changed during this time of pandemic learning, and there is a new challenge faced in education regarding how we effectively utilize the data now available to educators and leaders. In this session
Content provided by Microsoft
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Accelerate Learning with Project-Based Learning
Earlier this year, the George Lucas Educational Foundation released four new studies highlighting how project-based learning (PBL) helps accelerate student learning—across age groups, multiple disciplines, and different socio-economic statuses. With this year’s emphasis on unfinished
Content provided by SmartLab Learning
School & District Management Live Online Discussion Principal Overload: How to Manage Anxiety, Stress, and Tough Decisions
According to recent surveys, more than 40 percent of principals are considering leaving their jobs. With the pandemic, running a school building has become even more complicated, and principals' workloads continue to grow. If we

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management The Already Dire Substitute Shortage Could Get 'Worse Before It Gets Better'
School districts are trying all sorts of tactics, including increasing pay and relaxing requirements, to get more subs in classrooms.
10 min read
Image of an empty classroom.
urfinguss/iStock/Getty
School & District Management Opinion National School Boards Association Chooses to Be Part of the Problem
The NSBA chose to blur the distinction between permissible and suspect speech in suggesting the FBI should target unruly protesters.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School & District Management Facing Disruption and Firings, L.A. Extends COVID Vaccine Deadline for School Staff
The extension comes as the nation's second-largest school system has struggled to fill more than 2,000 teaching and other vacancies.
Howard Blume, Los Angeles Times
5 min read
In this March 2, 2021, file photo, a sign is displayed at a COVID-19 vaccination site for employees of the Los Angeles School District, LAUSD, in the parking lot of SOFI Stadium in Inglewood, Calif. Public schools have struggled for years with teacher shortages, particularly in math, science, special education and languages. But the pandemic has exacerbated the problem. The stresses of teaching in the COVID-era caused a spike in teacher retirements and resignations. On top of that, schools now have to hire all kinds of additional staff, like tutors and special aides to help kids make up for learning losses, and more teachers to run online school for those not ready to return.
In this March 2, 2021, file photo, a sign is displayed at a COVID-19 vaccination site for employees of the Los Angeles School District, LAUSD, in the parking lot of SOFI Stadium in Inglewood, Calif.
Marcio Jose Sanchez/AP Photo
School & District Management Opinion Graduation Must Depend on Learning, Not Time
We’re long overdue to redesign our education system around competency, argue six superintendents. Here’s what that could look like.
Morcease Beasley, Alberto Carvalho, William Hite, Jesus Jara, Monica Goldson & Jerry Almendarez
5 min read
A conceptual illustration of a mountain of paperwork before the goal is reached.
Vanessa Solis/Education Week and iStock