In 1997, the Mexican government launched a bold anti-poverty experiment in seven poor, rural states. The government’s idea was to give cash payments to mothers who met two key conditions: They had to enroll their children in school and take them for regular health checkups.
The program, known by the Spanish acronym PROGRESA, worked. In 18 months, school enrollments increased, families reported eating more nutritious meals, and children grew healthier. What was just as remarkable was that Mexican authorities could document those improvements through a massive, randomized experiment that involved 24,000 households and eminent researchers from around the world.
“It affected some people’s attitudes about randomized experiments like the Tennessee STAR study did in the United States,” said Patrick J. McEwan, an economist at Wellesley College in Wellesley, Mass. He referred to a landmark class-size experiment, called Student/Teacher Achievement Ratio or STAR, that touched off a spate of state and national programs in the 1990s that were aimed at creating smaller classes in the early grades.
“People suddenly realized, ‘My God, this one thing has taught us so much more than so many other studies have,’ ” Mr. McEwan added.
Researchers are beginning to learn a great deal, in fact, from dozens of randomized education studies going on now throughout the developing world. Spurred in part by PROGRESA’s success, researchers are using the methodology to evaluate programs for reducing class sizes in Kenya, providing tuition vouchers in Colombia and Chile, and decreasing teacher absenteeism and raising student achievement in rural schools in India, among other projects.
Considered by some experts to be the “gold standard” for determining what works, such studies are experiments in which participants are randomly assigned to either a control or a treatment group.
“To be able to say this intervention led to these gains gives us a lot more credibility and confidence to expand that program in that country,” said Harry A. Patrinos, a lead education economist for the World Bank, which has been among the international groups seeking to boost the number of randomized studies in its work portfolio in recent years.
Outside Help
Yet, while such trials transformed medical practice in this country and around the world, the methodology has been slower to take hold in education—partly because such studies are difficult and partly because of the ethical questions that arise when a promising intervention is given to some children but not others.
The same sorts of obstacles crop up in the developing world, according to researchers. But what has helped smooth the path for randomized trials in some of those countries has been the involvement of nongovernmental organizations willing to underwrite, or help carry out, education studies that they hope will deliver promising strategies for alleviating poverty.
Learn more about the Abdul Latif Jameel Poverty Action Lab.
“It’s easier for those organizations to innovate, and they’re more willing to try things out,” said Esther Duflo, a co-founder of the Abdul Latif Jameel Poverty Action Lab at the Massachusetts Institute of Technology, in Cambridge, Mass. “These organizations are not mandated like governments to serve everyone, so randomization is fine.”
The lab, made famous by the rock star Bono’s anti-AIDS and anti-poverty campaigns, was established three years ago to foster more social experimentation in developing countries. Its affiliated investigators are running 30 experiments in such nations around the globe. While most of that work has been in partnership with nongovernmental groups, lab researchers say they’re working with some government agencies as well.
“In a lot of these countries, they think, ‘What is scientific? What is modern?’ ” said Caroline M. Hoxby, a Harvard University economist whose graduate students have undertaken studies abroad. “If you talk to a policymaker in Indonesia or India or Africa, they’re often really interested in randomized-control trials because they really want to know what works.”
But developing nations can also present their own share of obstacles to experimentation.
Eric P. Bettinger is a researcher who was part of a team studying the effects of a school choice program in Colombia. Because the program was oversubscribed, government officials used a lottery system to decide which children would be given vouchers to attend private schools. He said the study team encountered problems trying to maintain the purity of the experiment because the local governments in some areas wanted to use the vouchers as a bargaining chip with guerrilla groups.
In other areas, records were lost or destroyed. In the end, the investigators wound up with just two or three sites with usable data, according to Mr. Bettinger, who is now an associate professor of economics at Case Western Reserve University, in Cleveland.
Different Contexts
Ms. Duflo said educators, policymakers, and nongovernmental agencies in the developing world turned to researchers for help when it became clear that studies conducted in the United States and other developed nations had little to offer developing countries’ educators and policymakers.
The contexts of the studies were too different. For instance, the researchers studying the effects of smaller classes in Tennessee measured what happened when classes were reduced from an average of 20 or 25 students to 15. In comparison, classes in Kenya can average 100 students. Likewise, rural Indian and East African schools grapple with teacher absenteeism rates that average more than 20 percent—much higher than would typically be the case in the United States.
“And sometimes, the biggest impact might come from something that’s a surprise,” Ms. Duflo said. For instance, when lab investigators evaluated a health program in Kenya, they stumbled on a strategy for boosting attendance that was even more cost-effective than the PROGRESA project: free deworming treatments for schoolchildren.
“You make children less sick and they come to school more,” Ms. Duflo explained.
For the same reason, it’s hard to gauge how much U.S. educators can learn from the stepped-up interest in randomized education research in developing nations. Findings from a deworming experiment in Kenya may have little to offer American educators, but experiments with vouchers and teacher-incentive programs in Colombia, where the education structure is slightly more developed, might possibly inform policy and practice.
“One could argue that humans are humans,” said Karthik Muralidharan, a graduate economics student at Harvard University. He’s spearheading an experiment in a rural province of India to test the effects of teacher incentives in 500 schools. Teachers in one group of 100 schools were told they could qualify for individual cash bonuses if they succeeded in raising their pupils’ test scores. In another group of 100 schools, the same bonuses would go to groups of teachers.
Mr. Muralidharan hopes to compare the test scores that result with those for three other control groups. The other groups are 100 schools given the same sum of money to hire teachers’ aides, 100 schools given the same amount in a no-strings-attached block grant to spend on learning needs, and a business-as-usual group. The final results aren’t yet complete, but the interventions under Mr. Muralidharan’s microscope bear echoes of national policy debates in the United States.
“Let’s suppose you can’t find a good, randomized experiment that’s going to shed light on what it is you want to do from the U.S.,” said Mr. Bettinger of Case Western. “Then you’re going to have to put your trust in two things—either observational studies or randomized experiments in other settings.”
