Published Online: September 15, 2009
Published in Print: September 16, 2009, as Randomized Trials: A Way To Stop 'Spinning Wheels'


Randomized Trials: A Way To Stop 'Spinning Wheels'

Article Tools
  • PrintPrinter-Friendly
  • EmailEmail Article
  • ReprintReprints
  • CommentsComments

To the Editor:

Although we support Lisbeth B. Schorr’s call for a variety of evaluation methods to identify promising social programs ("Innovative Reforms Require Innovative Scorekeeping," Aug. 26, 2009.), we respectfully disagree with her rejection of a central role for randomized controlled trials.

Our reasoning—to quote a recent National Academy of Sciences recommendation—is that evidence of effectiveness generally cannot be considered definitive without ultimate confirmation in well-conducted randomized trials, “even if based on the next strongest designs.” In fact, the history of social policy and medicine is replete with interventions that appeared highly promising in preliminary studies, but were subsequently found ineffective or even harmful in well-conducted trials.

Ms. Schorr’s advocacy of innovative programs without definitive evaluations is the approach U.S. social policy has largely followed, with little to show. New programs, introduced with fanfare as able to produce dramatic gains, have come and gone, with no one knowing for sure which were effective, and minimal progress being made. The National Assessment of Educational Progress’ long-term-trend data, for example, show little improvement in K-12 education achievement since the 1970s. Similarly, the official U.S. poverty rate, 12.5 percent, is slightly higher than it was in 1973.

Randomized trials offer a way to end this spinning of wheels.

Contrary to Ms. Schorr’s statement that they cannot evaluate “complex social programs … with multiple components,” a number of trials have done just that, producing valid, actionable evidence. For example, there have been randomized evaluations of comprehensive school reforms (such as those of the Success For All Foundation and the Comer School Development Program); communitywide underage-drinking prevention (Project Northland); countywide prevention of child maltreatment (Triple P); and multicomponent antipoverty strategies (PROGRESA).

In fact, PROGRESA—evaluated in 500 Mexican communities—showed sizable improvements in child education and health, was therefore implemented nationwide, and spawned replications in the United States and elsewhere accompanied by randomized evaluations. It illustrates how evidence-based policy can spark true progress.

Jon Baron
Coalition for Evidence-Based Policy
Washington, D.C.

Vol. 29, Issue 03, Page 32

Related Stories
You must be logged in to leave a comment. Login | Register
Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.

Back to Top Back to Top

Most Popular Stories