Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
School & District Management

International Research Group Meets to Review Its Efforts

By Debra Viadero — February 25, 2004 4 min read

Thoughtful practitioners often become confused when they look to the research evidence to back up interventions aimed at preventing crime, reducing poverty, or improving schools. One study says a program works. Another maintains it doesn’t.

That’s why the Campbell Collaboration was born four years ago. Made up of scientists, statisticians, and policymakers from around the world, the group’s aim is to shepherd systematic reviews of studies and distill nuggets of reliable information practitioners could use.

Last week, the group got an early glimpse at the nuggets its efforts are producing. More than 250 practitioners and researchers from 16-plus countries came to the nation’s capital to hear “reviews of reviews” of studies on teenage-pregnancy prevention, justice and anti-crime programs, peer-assisted learning, block scheduling, after-school programs, and other interventions.

Most of the reviews have not yet earned the group’s official stamp of approval. But, by showcasing the studies in its pipeline and others, the group is hoping to convince skeptics that its approach is both feasible and potentially useful.

“There are challenges, but I think the growing message from the field is that these are doable,” said Susan Goerlich Zief, a doctoral student at the University of Pennsylvania in Philadelphia and the co-author of a study discussed here.

Ms. Zief’s pilot study of after-school programs, technically called a “test-bed” review, focused only on randomized controlled trials—in other words, studies in which subjects are randomly assigned to either experimental or comparison groups. Of the eight such studies found by Ms. Zief and her research partner, Sherri Lauver, five met their review criteria.

Once they synthesized the studies’ effects, the researchers concluded that after-school programs may be a wash when it comes to improving students’ grades or reducing the number of hours they spend unsupervised.

“Do we want to say these programs don’t work?” asked Ms. Lauver.

Not yet. Like many of the researchers who presented preliminary work here, the Pennsylvania team said many factors could explain the null findings. Such factors include: too few rigorous studies, varied levels of program implementation, differences in the way studies measured the same outcomes, and differences among the programs themselves.

“We’re learning a lot,’' said Robert F. Boruch, Campbell’s co-chairman. “It’s going to take time to get this stuff pumped out.”

Gauging Effects

Other study reviews presented at the meeting also suggest that:

  • Block scheduling has “no harmful effect” on high school students’ academic achievement in the short run. That conclusion comes from a review of seven studies by Chance W. Lewis and Marc A. Winokur, both researchers from Colorado State University in Fort Collins. They added that schools might have reasons other than bolstering achievement for implementing those programs.
  • Peer-assisted learning, a process involving students in collaborative, group learning, can produce learning gains among elementary pupils—particularly those who are poor or members of minority groups. In a study first published last year, researchers Marika D. Ginsburg-Block of the University of Delaware in Newark and Cynthia A. Rohrbeck of George Washington University here base that conclusion on an analysis of 81 studies.
  • Teenage-pregnancy-prevention programs seem to have little effect on reducing pregnancy rates, delaying sexual initiation, or promoting contraceptive use. For her pilot review, Lauren Scher of the University of Pennsylvania analyzed 41 randomized studies. But she, too, said policymakers shouldn’t interpret her findings to mean such programs won’t work.

‘Informed Choices’

Organizers were not discouraged that so many of the findings seemed to show no effect. They noted that the same happened with early reviews from the Cochrane Collaboration, the medical- research group on which the Campbell Collaboration is modeled.

The going may also be slow, conference-goers said, because the rigorous “meta-analyses” the group is hoping to produce can be challenging, time-consuming, and hard to get funded.

That so many of the reviews here focused on education, however, reflects the high priority Washington policymakers and some foundations have put on generating “scientifically based research” in the field. The Campbell group, in fact, is also a partner in developing the U.S. Department of Education’s What Works Clearinghouse, a parallel effort to vet research for the field.

Whether the group’s work will translate into changes in social policy remains to be seen. Successive Campbell reviews of “Scared Straight’’ programs, which use convicts to describe harsh truths about prison life to potential juvenile offenders, suggest those programs may encourage a life of crime, rather than deter it. Even so, Illinois lawmakers voted last year to continue the program in their state.

“I don’t know that anyone would realistically expect any of this would lead policymakers to shut down a program and start a whole new set of programs,” said Dennis W. Cheek, a spokesman for the collaboration. “But we would hope that they at least would make informed choices.”

Related Tags:

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Learn directly from the pros why K-12 branding and marketing matters, and how to do it effectively.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Join experts from Samsung and Boxlight to learn how to make learning more interactive from anywhere.
Content provided by Samsung
Teaching Live Online Discussion A Seat at the Table With Education Week: How Educators Can Respond to a Post-Truth Era
How do educators break through the noise of disinformation to teach lessons grounded in objective truth? Join to find out.

EdWeek Top School Jobs

Director of Information Technology
Montpelier, Vermont
Washington Central UUSD
Great Oaks AmeriCorps Fellow August 2021 - June 2022
New York City, New York (US)
Great Oaks Charter Schools
Director of Athletics
Farmington, Connecticut
Farmington Public Schools
Head of Lower School
San Diego, California
San Diego Jewish Academy

Read Next

School & District Management Student Mental Health and Learning Loss Continue to Worry Principals
Months into the pandemic, elementary principals say they still want training in crucial areas to help students who are struggling.
3 min read
Student sitting alone with empty chairs around her.
Maria Casinos/iStock/Getty
School & District Management Opinion A Road Map for Education Research in a Crisis
Here are five basic principles for a responsible and timely research agenda during the COVID-19 pandemic.
Robin J. Lake
4 min read
Two opposing sides reaching out to work together
J.R. Bee for Education Week
School & District Management 1,000 Students, No Social Distancing, and a Fight to Keep the Virus Out
A principal describes the "nightmare" job of keeping more than 1,000 people safe in the fast-moving pandemic.
4 min read
Dixie Rae Garrison, principal of West Jordan Middle School, in West Jordan, Utah.
Dixie Rae Garrison, principal of West Jordan Middle School in West Jordan, Utah, would have preferred a hybrid schedule and other social distancing measures.
Courtesy of Dixie Rae Garrison
School & District Management A School Leader Who Calls Her Own Shots on Battling the Coronavirus
A charter school founder uses her autonomy to move swiftly on everything from classroom shutdowns to remote schooling.
3 min read
Nigena Livingston, founder and head of School at the URBAN ACT Academy in Indianapolis, Ind.
Nigena Livingston, founder and head of school at the URBAN ACT Academy in Indianapolis, makes swift decisions in responding to the threat of COVID-19 in her school community.
Courtesy of Nigena Livingston