Opinion
School & District Management Opinion

Making Policy Work: The Lesson From Medicine

By Jon Baron — May 22, 2007 6 min read
  • Save to favorites
  • Print

Two blockbuster research findings reported recently in the national press—one from the field of education, the other from medicine—have something important in common: They are the latest cases in which widely used, widely accepted practices have been challenged by scientifically rigorous evaluations. The education study found no difference in academic achievement between students who used leading educational software for reading and math in their classrooms, and those taught by other methods. (“Major Study on Software Stirs Debate,” April 11, 2007.) The medical study, reported in late March, found that stents, which are widely used to open clogged arteries, unexpectedly do no better than drugs for most heart patients.

— David Cutler

BRIC ARCHIVE

These two studies, although similar in methodology, were conducted in very different policy environments. In medicine, such rigorous evaluations are common, and often drive policy and practice. By contrast, in education and most other areas of social policy, such studies are relatively rare. In these areas, policy and practice tend to be driven more by advocacy studies and anecdote than by rigorous evidence, costing billions of dollars yet often failing to produce meaningful improvements in educational achievement, employment and earnings, or rates of substance abuse and criminal behavior, for example.

There is strong reason to believe education would benefit greatly from a more evidence-based approach, such as that used in medicine.

Both of the studies cited above were well-designed randomized-control trials, widely considered the gold standard for determining “what works” in medicine. Such trials—in which individuals are randomly assigned to a treatment group or a control group—have stunned the medical community before by overturning both conventional wisdom and the results of less rigorous studies. Examples include hormone-replacement therapy for postmenopausal women (shown to increase the risk of coronary heart disease, stroke, and breast cancer), dietary fiber to prevent colon cancer (shown to have no effect), and an oxygen-rich environment for premature infants (shown to increase blindness).

Well-designed trials have not only identified medical practices that are ineffective or harmful, they have also provided the conclusive evidence of effectiveness for most of the major medical advances over the past 50 years. These include, for example, vaccines for polio, measles, and hepatitis B; effective treatments for hypertension and high cholesterol; and cancer treatments that have dramatically improved survival rates from leukemia, Hodgkin’s disease, breast cancer, and many other cancers.

In the few cases where rigorous methods have been used in education, they have demonstrated the ability to produce valid, actionable evidence about what works, and the potential to spark rapid progress similar to that which has transformed medicine. For example, well-designed randomized-control trials in education have identified a few widely used programs that are ineffective or harmful. The trial of educational software products is one example, but there are others. Drug Abuse Resistance Education, or DARE—a substance-abuse-prevention program used in more than two-thirds of U.S. school districts—has been shown in such trials to have no effect on substance use, and therefore is now being redesigned. The Job Training Partnership Act program for young people—a large federal workforce-training program in the 1980s—was shown in a well-designed trial actually to have an adverse effect on the earnings of male youths. And a similar trial of federally funded dropout-prevention programs in the 1990s found no effect on school dropout rates.

Well-designed trials have also identified a few highly effective educational programs. One example is Success for All, a comprehensive schoolwide reform program, primarily for high-poverty elementary schools, with a strong emphasis on the prevention of reading problems before they become serious. A recent trial found that the average school implementing the program in grades K-2 scored higher on schoolwide reading achievement at the end of 2nd grade than approximately 60 percent of the schools in the control group.

Another example is the Good Behavior Game, a 1st grade classroom-management strategy that rewards students for positive group behavior. It has been shown in two well-designed trials in Baltimore public schools to produce 25 percent to 60 percent reductions in substance abuse, school suspensions, and serious conduct problems in youths through middle school and into young adulthood. And Check and Connect—a dropout-prevention program for at-risk high school students that assigns them a “monitor” (such as a graduate student) who serves as a year-round mentor and service coordinator—has been shown to be highly effective in two such trials, producing a 40 percent increase in students’ staying enrolled in or graduating from high school four years later.

The very existence of a few research-proven educational programs such as these suggests that a concerted government effort to apply the rigorous methods used in medicine to education policy could fundamentally increase the effectiveness of such policy in improving educational and life outcomes for American students. The U.S. Department of Education’s Institute of Education Sciences, or IES, has made an excellent start by greatly increasing the number of well-designed randomized-control trials and other rigorous evaluations it funds. But much more is needed, because the number of educational practices proven effective in such studies is very small, or even nonexistent in some areas of education. This leaves schools and districts with few research-proven tools they can use to increase the reading and math proficiency of students, as called for in the federal No Child Left Behind Act, or to improve other key educational outcomes.

In education, policy and practice tend to be driven more by advocacy studies and anecdote than by rigorous evidence.

An important step to address this problem has been proposed by the Aspen Institute’s bipartisan Commission on No Child Left Behind. That national panel has recommended doubling the IES research budget for K-12 education, currently about $250 million a year.

A second, complementary idea has been proposed by the National Board for Education Sciences, which oversees the research agenda of the IES. The board has recommended that Congress require federal education program grantees, as a condition of their grant awards, to participate in evaluations if asked to do so, including by random assignment where appropriate. This would help foster partnerships between program grantees and evaluators in carrying out rigorous evaluations to identify effective models and practices.

A third important step would be to create strong incentives for those who receive federal education grants to adopt research-proven models and practices in areas where such models or practices exist, and to provide assistance to put them into widespread use. A recent Education Week Commentary proposed a promising approach to advance this goal: a competitive priority for grant applicants that commit to the use of programs backed by strong evidence of effectiveness. (“Research and Effectiveness,” Oct. 18, 2006.)

A concerted effort to develop and use rigorous evidence to improve education policy, through steps such as these, would require a very modest investment of government funds in the context of the $400 billion spent each year on public K-12 education. Doubling the Institute of Education Sciences’ research budget, for instance, would cost, at an additional $250 million a year, only about 0.06 percent of the total K-12 expenditure, and the other two ideas above are budget-neutral. Yet they offer the opportunity, based on a compelling precedent from the field of medicine, to supply a critical missing piece needed to improve U.S. education: scientifically valid, actionable knowledge about what works in raising student achievement, preventing educational failure, and producing creative, motivated students who will be contributing members of society.

Related Tags:

A version of this article appeared in the May 23, 2007 edition of Education Week as Making Policy Work: The Lesson From Medicine

Events

English-Language Learners Webinar Helping English-Learners Through Improved Parent Outreach: Strategies That Work
Communicating with families is key to helping students thrive – and that’s become even more apparent during a pandemic that’s upended student well-being and forced constant logistical changes in schools. Educators should pay particular attention
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Q&A School Libraries and Controversial Books: Tips From the Front Lines
A top school librarian explains how districts can prepare for possible challenges to student reading materials and build trust with parents.
6 min read
Image of library shelves of books.
mikdam/iStock/Getty
School & District Management Opinion ‘This Is Not What We Signed Up For’: A Principal’s Plea for More Support
School leaders are playing the role of health-care experts, social workers, mask enforcers, and more. It’s taking a serious toll.
Kristen St. Germain
3 min read
Illustration of a professional woman walking a tightrope.
Laura Baker/Education Week and uzenzen/iStock/Getty
School & District Management Letter to the Editor Educators Must Look to History When They Advocate for Changes
Educators and policymakers must be aware of the history of ideas when making changes in education, says this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.
iStock/Getty
School & District Management Letter to the Editor Reconsidering Causes of Principal Burnout
The state and federal governments are asking us to implement policies that often go against our beliefs, says this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.
iStock/Getty