School & District Management

International Research Group Meets to Review Its Efforts

By Debra Viadero — February 25, 2004 4 min read

Thoughtful practitioners often become confused when they look to the research evidence to back up interventions aimed at preventing crime, reducing poverty, or improving schools. One study says a program works. Another maintains it doesn’t.

That’s why the Campbell Collaboration was born four years ago. Made up of scientists, statisticians, and policymakers from around the world, the group’s aim is to shepherd systematic reviews of studies and distill nuggets of reliable information practitioners could use.

Last week, the group got an early glimpse at the nuggets its efforts are producing. More than 250 practitioners and researchers from 16-plus countries came to the nation’s capital to hear “reviews of reviews” of studies on teenage-pregnancy prevention, justice and anti-crime programs, peer-assisted learning, block scheduling, after-school programs, and other interventions.

Most of the reviews have not yet earned the group’s official stamp of approval. But, by showcasing the studies in its pipeline and others, the group is hoping to convince skeptics that its approach is both feasible and potentially useful.

“There are challenges, but I think the growing message from the field is that these are doable,” said Susan Goerlich Zief, a doctoral student at the University of Pennsylvania in Philadelphia and the co-author of a study discussed here.

Ms. Zief’s pilot study of after-school programs, technically called a “test-bed” review, focused only on randomized controlled trials—in other words, studies in which subjects are randomly assigned to either experimental or comparison groups. Of the eight such studies found by Ms. Zief and her research partner, Sherri Lauver, five met their review criteria.

Once they synthesized the studies’ effects, the researchers concluded that after-school programs may be a wash when it comes to improving students’ grades or reducing the number of hours they spend unsupervised.

“Do we want to say these programs don’t work?” asked Ms. Lauver.

Not yet. Like many of the researchers who presented preliminary work here, the Pennsylvania team said many factors could explain the null findings. Such factors include: too few rigorous studies, varied levels of program implementation, differences in the way studies measured the same outcomes, and differences among the programs themselves.

“We’re learning a lot,’' said Robert F. Boruch, Campbell’s co-chairman. “It’s going to take time to get this stuff pumped out.”

Gauging Effects

Other study reviews presented at the meeting also suggest that:

  • Block scheduling has “no harmful effect” on high school students’ academic achievement in the short run. That conclusion comes from a review of seven studies by Chance W. Lewis and Marc A. Winokur, both researchers from Colorado State University in Fort Collins. They added that schools might have reasons other than bolstering achievement for implementing those programs.
  • Peer-assisted learning, a process involving students in collaborative, group learning, can produce learning gains among elementary pupils—particularly those who are poor or members of minority groups. In a study first published last year, researchers Marika D. Ginsburg-Block of the University of Delaware in Newark and Cynthia A. Rohrbeck of George Washington University here base that conclusion on an analysis of 81 studies.
  • Teenage-pregnancy-prevention programs seem to have little effect on reducing pregnancy rates, delaying sexual initiation, or promoting contraceptive use. For her pilot review, Lauren Scher of the University of Pennsylvania analyzed 41 randomized studies. But she, too, said policymakers shouldn’t interpret her findings to mean such programs won’t work.

‘Informed Choices’

Organizers were not discouraged that so many of the findings seemed to show no effect. They noted that the same happened with early reviews from the Cochrane Collaboration, the medical- research group on which the Campbell Collaboration is modeled.

The going may also be slow, conference-goers said, because the rigorous “meta-analyses” the group is hoping to produce can be challenging, time-consuming, and hard to get funded.

That so many of the reviews here focused on education, however, reflects the high priority Washington policymakers and some foundations have put on generating “scientifically based research” in the field. The Campbell group, in fact, is also a partner in developing the U.S. Department of Education’s What Works Clearinghouse, a parallel effort to vet research for the field.

Whether the group’s work will translate into changes in social policy remains to be seen. Successive Campbell reviews of “Scared Straight’’ programs, which use convicts to describe harsh truths about prison life to potential juvenile offenders, suggest those programs may encourage a life of crime, rather than deter it. Even so, Illinois lawmakers voted last year to continue the program in their state.

“I don’t know that anyone would realistically expect any of this would lead policymakers to shut down a program and start a whole new set of programs,” said Dennis W. Cheek, a spokesman for the collaboration. “But we would hope that they at least would make informed choices.”

Related Tags:

Events

Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How to Power Your Curriculum With Digital Books
Register for this can’t miss session looking at best practices for utilizing digital books to support their curriculum.
Content provided by OverDrive
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Embracing Student Engagement: The Pathway to Post-Pandemic Learning
As schools emerge from remote learning, educators are understandably worried about content and skills that students would otherwise have learned under normal circumstances. This raises the very real possibility that children will face endless hours
Content provided by Newsela

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion Are Your Leadership Practices Good Enough for Racial Justice?
Scratch being a hero. Instead, build trust and reach beyond school walls, write Jennifer Cheatham and John B. Diamond.
Jennifer Cheatham & John B. Diamond
5 min read
Illustration of leadership.
Collage by Laura Baker/Education Week (Images: DigitalVision Vectors, iStock, Getty)
School & District Management We Pay Superintendents Big Bucks and Expect Them to Succeed. But We Hardly Know Them
National data is skimpy, making it hard to know what influences superintendents' decisions to move on, retire, or how long they stay. Why?
8 min read
Conceptual image of tracking with data.
marcoventuriniautieri/iStock/Getty
School & District Management Data For the First Time in the Pandemic, a Majority of 4th Graders Learn in Person Full Time
The latest monthly federal data still show big racial and socioeconomic differences in who has access to full-time in-person instruction.
3 min read
Student with backpack.
surasaki/iStock/Getty
School & District Management From Our Research Center To Offer Remote Learning in the Fall or Not? Schools Are Split
An EdWeek Research Center survey shows that nearly 4 of every 10 educators say their schools will not offer any remote instruction options.
4 min read
Image of a teacher working with a student through a screen session.
Ridofranz/iStock/Getty