School & District Management

Randomized Trials Flourish in Developing Countries

By Debra Viadero — October 30, 2006 6 min read
  • Save to favorites
  • Print

In 1997, the Mexican government launched a bold anti-poverty experiment in seven poor, rural states. The government’s idea was to give cash payments to mothers who met two key conditions: They had to enroll their children in school and take them for regular health checkups.

The program, known by the Spanish acronym PROGRESA, worked. In 18 months, school enrollments increased, families reported eating more nutritious meals, and children grew healthier. What was just as remarkable was that Mexican authorities could document those improvements through a massive, randomized experiment that involved 24,000 households and eminent researchers from around the world.

“It affected some people’s attitudes about randomized experiments like the Tennessee STAR study did in the United States,” said Patrick J. McEwan, an economist at Wellesley College in Wellesley, Mass. He referred to a landmark class-size experiment, called Student/Teacher Achievement Ratio or STAR, that touched off a spate of state and national programs in the 1990s that were aimed at creating smaller classes in the early grades.

“People suddenly realized, ‘My God, this one thing has taught us so much more than so many other studies have,’ ” Mr. McEwan added.

Researchers are beginning to learn a great deal, in fact, from dozens of randomized education studies going on now throughout the developing world. Spurred in part by PROGRESA’s success, researchers are using the methodology to evaluate programs for reducing class sizes in Kenya, providing tuition vouchers in Colombia and Chile, and decreasing teacher absenteeism and raising student achievement in rural schools in India, among other projects.

Considered by some experts to be the “gold standard” for determining what works, such studies are experiments in which participants are randomly assigned to either a control or a treatment group.

“To be able to say this intervention led to these gains gives us a lot more credibility and confidence to expand that program in that country,” said Harry A. Patrinos, a lead education economist for the World Bank, which has been among the international groups seeking to boost the number of randomized studies in its work portfolio in recent years.

Outside Help

Yet, while such trials transformed medical practice in this country and around the world, the methodology has been slower to take hold in education—partly because such studies are difficult and partly because of the ethical questions that arise when a promising intervention is given to some children but not others.

The same sorts of obstacles crop up in the developing world, according to researchers. But what has helped smooth the path for randomized trials in some of those countries has been the involvement of nongovernmental organizations willing to underwrite, or help carry out, education studies that they hope will deliver promising strategies for alleviating poverty.

Learn more about the Abdul Latif Jameel Poverty Action Lab.

“It’s easier for those organizations to innovate, and they’re more willing to try things out,” said Esther Duflo, a co-founder of the Abdul Latif Jameel Poverty Action Lab at the Massachusetts Institute of Technology, in Cambridge, Mass. “These organizations are not mandated like governments to serve everyone, so randomization is fine.”

The lab, made famous by the rock star Bono’s anti-AIDS and anti-poverty campaigns, was established three years ago to foster more social experimentation in developing countries. Its affiliated investigators are running 30 experiments in such nations around the globe. While most of that work has been in partnership with nongovernmental groups, lab researchers say they’re working with some government agencies as well.

“In a lot of these countries, they think, ‘What is scientific? What is modern?’ ” said Caroline M. Hoxby, a Harvard University economist whose graduate students have undertaken studies abroad. “If you talk to a policymaker in Indonesia or India or Africa, they’re often really interested in randomized-control trials because they really want to know what works.”

But developing nations can also present their own share of obstacles to experimentation.

Eric P. Bettinger is a researcher who was part of a team studying the effects of a school choice program in Colombia. Because the program was oversubscribed, government officials used a lottery system to decide which children would be given vouchers to attend private schools. He said the study team encountered problems trying to maintain the purity of the experiment because the local governments in some areas wanted to use the vouchers as a bargaining chip with guerrilla groups.

In other areas, records were lost or destroyed. In the end, the investigators wound up with just two or three sites with usable data, according to Mr. Bettinger, who is now an associate professor of economics at Case Western Reserve University, in Cleveland.

Different Contexts

Ms. Duflo said educators, policymakers, and nongovernmental agencies in the developing world turned to researchers for help when it became clear that studies conducted in the United States and other developed nations had little to offer developing countries’ educators and policymakers.

The contexts of the studies were too different. For instance, the researchers studying the effects of smaller classes in Tennessee measured what happened when classes were reduced from an average of 20 or 25 students to 15. In comparison, classes in Kenya can average 100 students. Likewise, rural Indian and East African schools grapple with teacher absenteeism rates that average more than 20 percent—much higher than would typically be the case in the United States.

“And sometimes, the biggest impact might come from something that’s a surprise,” Ms. Duflo said. For instance, when lab investigators evaluated a health program in Kenya, they stumbled on a strategy for boosting attendance that was even more cost-effective than the PROGRESA project: free deworming treatments for schoolchildren.

“You make children less sick and they come to school more,” Ms. Duflo explained.

For the same reason, it’s hard to gauge how much U.S. educators can learn from the stepped-up interest in randomized education research in developing nations. Findings from a deworming experiment in Kenya may have little to offer American educators, but experiments with vouchers and teacher-incentive programs in Colombia, where the education structure is slightly more developed, might possibly inform policy and practice.

“One could argue that humans are humans,” said Karthik Muralidharan, a graduate economics student at Harvard University. He’s spearheading an experiment in a rural province of India to test the effects of teacher incentives in 500 schools. Teachers in one group of 100 schools were told they could qualify for individual cash bonuses if they succeeded in raising their pupils’ test scores. In another group of 100 schools, the same bonuses would go to groups of teachers.

Mr. Muralidharan hopes to compare the test scores that result with those for three other control groups. The other groups are 100 schools given the same sum of money to hire teachers’ aides, 100 schools given the same amount in a no-strings-attached block grant to spend on learning needs, and a business-as-usual group. The final results aren’t yet complete, but the interventions under Mr. Muralidharan’s microscope bear echoes of national policy debates in the United States.

“Let’s suppose you can’t find a good, randomized experiment that’s going to shed light on what it is you want to do from the U.S.,” said Mr. Bettinger of Case Western. “Then you’re going to have to put your trust in two things—either observational studies or randomized experiments in other settings.”

Related Tags:

Coverage of education research is supported in part by a grant from the Spencer Foundation.
A version of this article appeared in the November 01, 2006 edition of Education Week as Randomized Trials Flourish in Developing Countries

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management The Eclipse Is Great for Learning. But It's Tough on School Logistics
A total solar eclipse will cross a large swath of the country on April 8, sparking tough management choices for leaders of the districts in its path.
5 min read
A woman and stands outside with her arm on the back of a boy as they look up at the sky while wearing special paper glasses made for viewing a solar eclipse.
Jackie Johnson and her son Bradley Johnson, 9, watch a partial solar eclipse at the Frost Science Museum on Oct. 14, 2023, in downtown Miami. In 2024, some districts are planning to delay or cancel school on the day of a total eclipse, out of safety concerns.
Matias J. Ocner/Miami Herald via AP
School & District Management Opinion A Good Principal Knows When It's Time to Leave
I didn’t leave my job because of burnout; I stepped away from being a school leader because it was in everybody’s best interest.
Matthew Ebert
4 min read
Conceptual illustration of someone handing off a baton to someone else over a completed puzzle.
Vanessa Solis/Education Week via Canva
School & District Management Principals Tell Politicians on Capitol Hill: We’re Burning Out
Students' mental health top principals' growing list of concerns.
6 min read
People walk outside the U.S Capitol building in Washington, June 9, 2022.
Visitors walk outside the U.S Capitol building in Washington on June 9, 2022.
Patrick Semansky/AP
School & District Management Women Superintendents Experience Bias on the Climb to Leadership
Interpersonal slights and inequities make it hard for women to land the job and stay in it.
3 min read
Woman stands in front of a staircase in different colors. She is about to walk up the stairs. Concept of standing in front of a challenge and finding the right solution and courage to move on.
mikkelwilliam/E+