School & District Management

Randomized Trials Flourish in Developing Countries

By Debra Viadero — October 30, 2006 6 min read
  • Save to favorites
  • Print

In 1997, the Mexican government launched a bold anti-poverty experiment in seven poor, rural states. The government’s idea was to give cash payments to mothers who met two key conditions: They had to enroll their children in school and take them for regular health checkups.

The program, known by the Spanish acronym PROGRESA, worked. In 18 months, school enrollments increased, families reported eating more nutritious meals, and children grew healthier. What was just as remarkable was that Mexican authorities could document those improvements through a massive, randomized experiment that involved 24,000 households and eminent researchers from around the world.

“It affected some people’s attitudes about randomized experiments like the Tennessee STAR study did in the United States,” said Patrick J. McEwan, an economist at Wellesley College in Wellesley, Mass. He referred to a landmark class-size experiment, called Student/Teacher Achievement Ratio or STAR, that touched off a spate of state and national programs in the 1990s that were aimed at creating smaller classes in the early grades.

“People suddenly realized, ‘My God, this one thing has taught us so much more than so many other studies have,’ ” Mr. McEwan added.

Researchers are beginning to learn a great deal, in fact, from dozens of randomized education studies going on now throughout the developing world. Spurred in part by PROGRESA’s success, researchers are using the methodology to evaluate programs for reducing class sizes in Kenya, providing tuition vouchers in Colombia and Chile, and decreasing teacher absenteeism and raising student achievement in rural schools in India, among other projects.

Considered by some experts to be the “gold standard” for determining what works, such studies are experiments in which participants are randomly assigned to either a control or a treatment group.

“To be able to say this intervention led to these gains gives us a lot more credibility and confidence to expand that program in that country,” said Harry A. Patrinos, a lead education economist for the World Bank, which has been among the international groups seeking to boost the number of randomized studies in its work portfolio in recent years.

Outside Help

Yet, while such trials transformed medical practice in this country and around the world, the methodology has been slower to take hold in education—partly because such studies are difficult and partly because of the ethical questions that arise when a promising intervention is given to some children but not others.

The same sorts of obstacles crop up in the developing world, according to researchers. But what has helped smooth the path for randomized trials in some of those countries has been the involvement of nongovernmental organizations willing to underwrite, or help carry out, education studies that they hope will deliver promising strategies for alleviating poverty.

Learn more about the Abdul Latif Jameel Poverty Action Lab.

“It’s easier for those organizations to innovate, and they’re more willing to try things out,” said Esther Duflo, a co-founder of the Abdul Latif Jameel Poverty Action Lab at the Massachusetts Institute of Technology, in Cambridge, Mass. “These organizations are not mandated like governments to serve everyone, so randomization is fine.”

The lab, made famous by the rock star Bono’s anti-AIDS and anti-poverty campaigns, was established three years ago to foster more social experimentation in developing countries. Its affiliated investigators are running 30 experiments in such nations around the globe. While most of that work has been in partnership with nongovernmental groups, lab researchers say they’re working with some government agencies as well.

“In a lot of these countries, they think, ‘What is scientific? What is modern?’ ” said Caroline M. Hoxby, a Harvard University economist whose graduate students have undertaken studies abroad. “If you talk to a policymaker in Indonesia or India or Africa, they’re often really interested in randomized-control trials because they really want to know what works.”

But developing nations can also present their own share of obstacles to experimentation.

Eric P. Bettinger is a researcher who was part of a team studying the effects of a school choice program in Colombia. Because the program was oversubscribed, government officials used a lottery system to decide which children would be given vouchers to attend private schools. He said the study team encountered problems trying to maintain the purity of the experiment because the local governments in some areas wanted to use the vouchers as a bargaining chip with guerrilla groups.

In other areas, records were lost or destroyed. In the end, the investigators wound up with just two or three sites with usable data, according to Mr. Bettinger, who is now an associate professor of economics at Case Western Reserve University, in Cleveland.

Different Contexts

Ms. Duflo said educators, policymakers, and nongovernmental agencies in the developing world turned to researchers for help when it became clear that studies conducted in the United States and other developed nations had little to offer developing countries’ educators and policymakers.

The contexts of the studies were too different. For instance, the researchers studying the effects of smaller classes in Tennessee measured what happened when classes were reduced from an average of 20 or 25 students to 15. In comparison, classes in Kenya can average 100 students. Likewise, rural Indian and East African schools grapple with teacher absenteeism rates that average more than 20 percent—much higher than would typically be the case in the United States.

“And sometimes, the biggest impact might come from something that’s a surprise,” Ms. Duflo said. For instance, when lab investigators evaluated a health program in Kenya, they stumbled on a strategy for boosting attendance that was even more cost-effective than the PROGRESA project: free deworming treatments for schoolchildren.

“You make children less sick and they come to school more,” Ms. Duflo explained.

For the same reason, it’s hard to gauge how much U.S. educators can learn from the stepped-up interest in randomized education research in developing nations. Findings from a deworming experiment in Kenya may have little to offer American educators, but experiments with vouchers and teacher-incentive programs in Colombia, where the education structure is slightly more developed, might possibly inform policy and practice.

“One could argue that humans are humans,” said Karthik Muralidharan, a graduate economics student at Harvard University. He’s spearheading an experiment in a rural province of India to test the effects of teacher incentives in 500 schools. Teachers in one group of 100 schools were told they could qualify for individual cash bonuses if they succeeded in raising their pupils’ test scores. In another group of 100 schools, the same bonuses would go to groups of teachers.

Mr. Muralidharan hopes to compare the test scores that result with those for three other control groups. The other groups are 100 schools given the same sum of money to hire teachers’ aides, 100 schools given the same amount in a no-strings-attached block grant to spend on learning needs, and a business-as-usual group. The final results aren’t yet complete, but the interventions under Mr. Muralidharan’s microscope bear echoes of national policy debates in the United States.

“Let’s suppose you can’t find a good, randomized experiment that’s going to shed light on what it is you want to do from the U.S.,” said Mr. Bettinger of Case Western. “Then you’re going to have to put your trust in two things—either observational studies or randomized experiments in other settings.”

Related Tags:

Coverage of education research is supported in part by a grant from the Spencer Foundation.
A version of this article appeared in the November 01, 2006 edition of Education Week as Randomized Trials Flourish in Developing Countries


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Ensuring Continuity of Learning: How to Prepare for the Next Disruption
Across the country, K-12 schools and districts are, again, considering how to ensure effective continuity of learning in the face of emerging COVID variants, politicized debates, and more. Learn from Alexandria City Public Schools superintendent
Content provided by Class
Teaching Profession Live Online Discussion What Have We Learned From Teachers During the Pandemic?
University of California, Santa Cruz, researcher Lora Bartlett and her colleagues spent months studying how the pandemic affected classroom teachers. We will discuss the takeaways from her research not only for teachers, but also for

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management SEL for Principals: How a Professional Development Program Serves Their High-Stress Needs
A statewide program in Massachusetts guides principals on how to apply social-emotional learning and self-care skills to their own jobs.
10 min read
Image of a professional male in meditation pose.
iStock/Getty
School & District Management Some Teachers Won't Get Vaccinated, Even With a Mandate. What Should Schools Do About It?
Vaccine requirements for teachers are gaining traction, but the logistics of upholding them are complicated.
9 min read
Illustration of a vaccine, medical equipment, a clock and a calendar with a date marked in red.
iStock/Getty
School & District Management A Vaccine for Kids Is Coming. 6 Tips for Administering the Shot in Your School
Start planning now, get help, and build enthusiasm. It's harder than it looks.
11 min read
Cole Rodriguez, a 15-year-old student at Topeka West, gets a COVID-19 vaccine Monday, Aug. 9, 2021 at Topeka High School's vaccine clinic.
Cole Rodriguez, a 15-year-old student, gets a COVID-19 vaccine at Topeka High School's vaccine clinic.
Evert Nelson/The Topeka Capital-Journal via AP
School & District Management Letter to the Editor School Mask Mandates: Pandemic, ‘Panicdemic,’ or Personal?
"A pandemic is based on facts. A 'panicdemic' is based on fears. Today, we have both," writes a professor.
1 min read