When it comes to guiding principles, research in education is not all that different from research in the natural or social sciences, a panel of prominent researchers concludes in a new report.
Read “Scientific Inquiry in Education,” produced by the National Research Council and published by the National Academy Press.
Produced by a committee of the National Research Council, the report was put together partly in response to debates taking place among researchers and policymakers on the meaning of “scientifically based” research in education, a field often criticized for the uneven quality of its studies.
Some experts argue that the “gold standard” for education research should be strictly controlled, random-assignment studies. Others suggest that the methods of inquiry in the field can rightly be considerably broader, encompassing everything from classical experiments to well-written novels.
In its report, which was released last week, the 16-member panel favors neither the strictest nor the broadest position. Rather, it argues that studies in education, like studies in biology, physics, anthropology, and other areas, should pose significant questions, link to relevant theory, and use research tools that are appropriate for answering those questions.
Researchers should also systematically rule out counter-explanations, replicate their findings, and put their work and their data out to be scrutinized by their colleagues and the public, the report says.
“It’s not overly shocking, but it certainly sends the signal that anything doesn’t go,” Richard J. Shavelson, the Stanford University education professor who chaired the panel, said of the stance the group takes.
Sorting Out Science
While education researchers should do more randomized experiments, Mr. Shavelson said, the methodology might not be appropriate, for example, for studying the impact of teacher salaries on student achievement or for tracing young children’s language development.
At the same time, the report notes, some popular forms of education studies, such as “portraiture,” a method in which the researcher and subject collaborate in writing literary-style portraits, are not science.
|
Such work, said Robert F. Boruch, a University of Pennsylvania professor of education and statistics who served on the panel, “is a way of enlarging the way we think about phenomena in the same way an artist enlarges the way we think about color and perspective, but it’s not particularly science.”
The panel’s report is as notable for its diverse membership as it is for its findings. In addition to Mr. Boruch, who champions randomized experiments, the group includes an anthropologist, a cell biologist, an education historian, an economist, a reading-disabilities researcher, a chemistry professor, and a testing expert.
Convened a year ago, the panel met monthly in order to finish before Congress took up legislation to reauthorize the primary federal research office for education, the Department of Education’s office of educational research and improvement. As part of that process, some federal lawmakers last year took at stab at writing into law a definition of scientific education research—an attempt that many education researchers criticized as being too restrictive.
“We certainly hope that our report makes it unnecessary for federal legislators to say what they think science research is,” Mr. Shavelson said.
The report is also timely because President Bush and members of his administration have expressed a keen interest in promoting “scientifically based research” in education.
To encourage the kind of “scientific culture” in the OERI that might foster rigorous research, the report recommends insulating the agency’s research from political interference. It stops short, however, of describing how that might happen.
“You can’t do good research if people in Congress or in the executive branch are constantly coming and telling you what to do,” said panel member Ellen Condliffe Lagemann. She is president of the Chicago-based Spencer Foundation, which promotes education research, including coverage of the subject in Education Week.
Echoing recent reports on education research, the report says that federal support for education research is weak— and getting weaker. While total funding for the research agency grew tenfold from 1980 to 2000, the percentage of those dollars supporting studies fell sharply in the 1980s and now hovers around 15 percent.
The panelists also called for recruiting more experienced researchers to staff the agency and to review grant proposals.
Kenji Hakuta, the Stanford University education professor who chairs the National Educational Research Policy and Priorities Board, called the study good advice. His board, which advises the Education Department on its research operations, commissioned the $600,000 study.
“It says first and foremost the agency has to be populated by people who understand what the culture of science consists of,” he said, “and that the review of proposals needs to be managed through a peer-review process that reflects that culture.”
Department officials declined to comment on the report.
