Campbell Collaboration Seeks To Firm Up ‘Soft Sciences’

By Debra Viadero — April 03, 2002 7 min read
  • Save to favorites
  • Print

The pursuit of scientific inquiry has long known no national boundaries. So it’s not uncommon for similar studies on, say, an effective treatment for tuberculosis or a drug to combat breast cancer to turn up in any number of countries where scientists do their work.

It wasn’t until the 1970s, though, that a British epidemiologist named Archie Cochrane hit on the idea of gathering up all those studies, analyzing the findings, and distilling nuggets of scientific truth to guide the decisions that practitioners make in the field.

More information on The Campbell Collaboration is available at

Dr. Cochrane’s idea gave birth years later to the Cochrane Collaboration, an international organization based in the United Kingdom that systematically reviews findings from randomized experiments in health care. Since its 1993 founding, the group has made media splashes worldwide with reports on everything from the effectiveness of daily doses of aspirin in preventing heart disease to whether regular mammograms improve women’s cancer-survival rates.

Now, an international group of scientists and policymakers is working to do the same for the “soft sciences,” including education research. The aim of the Campbell Collaboration, which recently held its second annual meeting here at the University of Pennsylvania, is to find out what research has to say about the myriad interventions tried in the social sciences, and then translate those results into recommendations for policymakers, public administrators, educators, police agencies, and social workers.

Named for the late Donald T. Campbell, the American statistician who called for an “experimenting society,” the group formed three years ago with backing from foundations and government agencies.

“We’re trying to stand on the shoulders of the Cochrane Collaboration,” said Robert F. Boruch, the Penn professor who launched the Campbell group. “The emphasis is on evidence right now, and we’re in the right position to do something about it.”

Demanding Proof

The new interest in “evidence-based” social policy was apparent at the group’s February meeting. More than 200 researchers and policymakers from 16 countries showed up for the two-day event hosted by Penn’s graduate school of education—more than double the number that came to the group’s first official meeting, held in Stockholm.

The collaboration is gathering steam just as policymakers on both sides of the Atlantic Ocean are stepping up their demands for empirical proof that programs or approaches work.

Perhaps nowhere is that emphasis more apparent than in the United States, where such pressure increasingly is being felt in education. Federal lawmakers laced the new “No Child Left Behind” Act of 2001 with phrases such as “scientifically based research” and required states and school districts to rely on research in choosing, for example, school improvement programs and professional-development lessons.

In Britain, Prime Minister Tony Blair, who was elected after promising to build an “evidence-based” government, has even formed a Cabinet-level agency to guide that effort. And the Swedish government last year launched an initiative to better incorporate empirical knowledge into social work practice.

“If you look across the European scene over the last two years, there is a growing interest and conscientiousness among practitioners, policymakers, and researchers to make decisions and practices that are more research-based,” said Haluk Soydan, a social work professor from Sweden who co-chairs the Campbell steering group with Mr. Boruch.

And although cultures vary from nation to nation, what’s true for Sweden or Canada in the social sciences sometimes applies in other countries, too.

“Scientists say that similarities between a great many of our societies are augmenting rather than diminishing,” Mr. Soydan noted.

Like the Cochrane Collaboration, the Campbell Collaboration uses a statistical technique known as meta-analysis to synthesize findings. Traditionally, scientists looking for a research consensus on a particular intervention merely tallied the positive or negative findings from studies. But statistical experts said that method, known as “vote counting,” often yielded the same, lukewarm conclusion: “Results are inconsistent.”

“That’s where researchers shot themselves in the foot. They didn’t know how to synthesize studies, and constantly undersold what the research had to say to policymakers,” said Harris M. Cooper, a University of Missouri psychologist who heads Campbell’s methods group.

He said meta-analyses, in comparison, give a more finely grained look that takes into account the size of each individual effect, the consistency of those effects, and the confidence that researchers place on them.

To safeguard the objectivity of the reviews, researchers who take one on have to follow strict guidelines. Reviewers must ferret out every study ever done on a topic, whether published or not, as long as it meets the group’s criteria for a credible study. If reviewers have a potential bias toward a particular outcome, they have to reveal it, and they have to ensure that their funding comes from more than one source.

The group’s first review is a look at seven studies on “scared straight” programs. Used mostly in the United States, the programs enlist convicted criminals to give delinquent teenagers a harsh look at prison life, hoping to deter budding criminals from a life of crime.

It doesn’t, according to the review by Anthony J. Petrosino, a researcher with the American Academy of Arts and Sciences in Cambridge, Mass. His analysis suggests that scared-straight participants are more likely to rack up a subsequent arrest than nonparticipants.

Heeding Advice

The reviews of educational interventions, in comparison, will be slower in coming. Some of the hesitancy comes because those in the field have traditionally been wary of experiments and more quantitatively oriented studies. Some education researchers contend, for example, that it’s unethical to provide a potentially beneficial intervention to one group of children and not another. Others fear that researchers could “miss the trees for the forest” by focusing only on strict experiments with control groups and treatment groups.

For their part, the leaders of the Campbell Collaboration say they are not against qualitative research. They just haven’t figured out how to incorporate it.

“We know in these control trials we need to know what’s going on inside the black box— whether the cops are following orders or whether the initiatives in housing projects are really carried out,” Mr. Boruch said. “That’s where good qualitative people come in. The challenge is how to integrate it all.”

The collaboration’s education group is taking some cues for now from the Centre for Evidence-Informed Policy in Education at the University of London’s Institute of Education. With support from the British government, the center coordinates research syntheses—both qualitative and quantitative—on “what works” in education.

Its forthcoming reports focus on such topics as problem-based learning and volunteer tutoring, and Campbell’s creators are hoping to borrow some of them for their own archives.

In the meantime, the education group’s co-chairman, C. Kent McGuire, wants to prod others to do Campbell reviews. Mr. McGuire, the U.S. Department of Education’s assistant secretary for educational research and improvement under President Clinton, is also enlisting policymakers to decide what research questions the education group should undertake.

“When they don’t feel like they ‘own’ the questions, policymakers will not trust the evidence,” Mr. McGuire said.

Whether the politicians will indeed “trust the evidence” is still an open question for the fledgling group. As Philip Davies, the education panel’s other co-chairman, points out, so- called faith-based initiatives in social policy remain popular in both the United States and the United Kingdom even though their research base is shaky.

“I am not naive enough to assume that we’re going to get the British government or the U.S. government to give up some of their treasured beliefs,” said Mr. Davies, a policy evaluator in the agency that Prime Minister Blair set up to lead his push for evidence-based government. “What we’re trying to say is this is what the evidence tells us about what works. The test is for government to run with it or not, and see if they get re-elected.”

A version of this article appeared in the April 03, 2002 edition of Education Week as Campbell Collaboration Seeks To Firm Up ‘Soft Sciences’


English-Language Learners Webinar Helping English-Learners Through Improved Parent Outreach: Strategies That Work
Communicating with families is key to helping students thrive – and that’s become even more apparent during a pandemic that’s upended student well-being and forced constant logistical changes in schools. Educators should pay particular attention
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Nearly a Million Kids Vaccinated in Week 1, White House Says
Experts say there are signs that it will be difficult to sustain the initial momentum.
4 min read
Leo Hahn, 11, gets the first shot of the Pfizer COVID-19 vaccine, Tuesday, Nov. 9, 2021, at the University of Washington Medical Center in Seattle. Last week, U.S. health officials gave the final signoff to Pfizer's kid-size COVID-19 shot, a milestone that opened a major expansion of the nation's vaccination campaign to children as young as 5. (AP Photo/Ted S. Warren)
Education How Schools Are Getting Kids the COVID Shot, and Why Some Aren’t
Some district leaders say offering vaccine clinics, with the involvement of trusted school staff, is key to helping overcome hesitancy.
5 min read
A girl walks outside of a mobile vaccine unit after getting the first dose of her COVID-19 vaccine, outside P.S. 277, Friday, Nov. 5, 2021, in the Bronx borough of New York. (AP Photo/Eduardo Munoz Alvarez)
Education Biden Administration Urges Schools to Provide COVID-19 Shots, Information for Kids
The Biden administration is encouraging local school districts to host vaccine clinics for kids and information on benefits of the shots.
2 min read
President Joe Biden, and first lady Jill Biden walk to board Marine One on the South Lawn of the White House in Washington, Saturday, Nov. 6, 2021. Biden is spending the weekend at his home in Rehoboth Beach, Del. (AP Photo/Alex Brandon)
Education Civil Rights Groups Sue Tennessee Over Law Against Transgender Student Athletes
The state law bars transgender athletes from playing public high school or middle school sports aligned with their gender identity.
3 min read
Amy Allen, the mother of an 8th grade transgender son, speaks after a Human Rights Campaign round table discussion on anti-transgender laws in Nashville, Tenn. on May 21, 2021.
Amy Allen, the mother of an 8th grade transgender son, speaks after a Human Rights Campaign round table discussion on anti-transgender laws in Nashville, Tenn. on May 21, 2021.
Mark Humphrey/AP