Opinion
School & District Management Opinion

Making Policy Work: The Lesson From Medicine

By Jon Baron — May 22, 2007 6 min read
  • Save to favorites
  • Print

Two blockbuster research findings reported recently in the national press—one from the field of education, the other from medicine—have something important in common: They are the latest cases in which widely used, widely accepted practices have been challenged by scientifically rigorous evaluations. The education study found no difference in academic achievement between students who used leading educational software for reading and math in their classrooms, and those taught by other methods. (“Major Study on Software Stirs Debate,” April 11, 2007.) The medical study, reported in late March, found that stents, which are widely used to open clogged arteries, unexpectedly do no better than drugs for most heart patients.

— David Cutler

BRIC ARCHIVE

These two studies, although similar in methodology, were conducted in very different policy environments. In medicine, such rigorous evaluations are common, and often drive policy and practice. By contrast, in education and most other areas of social policy, such studies are relatively rare. In these areas, policy and practice tend to be driven more by advocacy studies and anecdote than by rigorous evidence, costing billions of dollars yet often failing to produce meaningful improvements in educational achievement, employment and earnings, or rates of substance abuse and criminal behavior, for example.

There is strong reason to believe education would benefit greatly from a more evidence-based approach, such as that used in medicine.

Both of the studies cited above were well-designed randomized-control trials, widely considered the gold standard for determining “what works” in medicine. Such trials—in which individuals are randomly assigned to a treatment group or a control group—have stunned the medical community before by overturning both conventional wisdom and the results of less rigorous studies. Examples include hormone-replacement therapy for postmenopausal women (shown to increase the risk of coronary heart disease, stroke, and breast cancer), dietary fiber to prevent colon cancer (shown to have no effect), and an oxygen-rich environment for premature infants (shown to increase blindness).

Well-designed trials have not only identified medical practices that are ineffective or harmful, they have also provided the conclusive evidence of effectiveness for most of the major medical advances over the past 50 years. These include, for example, vaccines for polio, measles, and hepatitis B; effective treatments for hypertension and high cholesterol; and cancer treatments that have dramatically improved survival rates from leukemia, Hodgkin’s disease, breast cancer, and many other cancers.

In the few cases where rigorous methods have been used in education, they have demonstrated the ability to produce valid, actionable evidence about what works, and the potential to spark rapid progress similar to that which has transformed medicine. For example, well-designed randomized-control trials in education have identified a few widely used programs that are ineffective or harmful. The trial of educational software products is one example, but there are others. Drug Abuse Resistance Education, or DARE—a substance-abuse-prevention program used in more than two-thirds of U.S. school districts—has been shown in such trials to have no effect on substance use, and therefore is now being redesigned. The Job Training Partnership Act program for young people—a large federal workforce-training program in the 1980s—was shown in a well-designed trial actually to have an adverse effect on the earnings of male youths. And a similar trial of federally funded dropout-prevention programs in the 1990s found no effect on school dropout rates.

Well-designed trials have also identified a few highly effective educational programs. One example is Success for All, a comprehensive schoolwide reform program, primarily for high-poverty elementary schools, with a strong emphasis on the prevention of reading problems before they become serious. A recent trial found that the average school implementing the program in grades K-2 scored higher on schoolwide reading achievement at the end of 2nd grade than approximately 60 percent of the schools in the control group.

Another example is the Good Behavior Game, a 1st grade classroom-management strategy that rewards students for positive group behavior. It has been shown in two well-designed trials in Baltimore public schools to produce 25 percent to 60 percent reductions in substance abuse, school suspensions, and serious conduct problems in youths through middle school and into young adulthood. And Check and Connect—a dropout-prevention program for at-risk high school students that assigns them a “monitor” (such as a graduate student) who serves as a year-round mentor and service coordinator—has been shown to be highly effective in two such trials, producing a 40 percent increase in students’ staying enrolled in or graduating from high school four years later.

The very existence of a few research-proven educational programs such as these suggests that a concerted government effort to apply the rigorous methods used in medicine to education policy could fundamentally increase the effectiveness of such policy in improving educational and life outcomes for American students. The U.S. Department of Education’s Institute of Education Sciences, or IES, has made an excellent start by greatly increasing the number of well-designed randomized-control trials and other rigorous evaluations it funds. But much more is needed, because the number of educational practices proven effective in such studies is very small, or even nonexistent in some areas of education. This leaves schools and districts with few research-proven tools they can use to increase the reading and math proficiency of students, as called for in the federal No Child Left Behind Act, or to improve other key educational outcomes.

In education, policy and practice tend to be driven more by advocacy studies and anecdote than by rigorous evidence.

An important step to address this problem has been proposed by the Aspen Institute’s bipartisan Commission on No Child Left Behind. That national panel has recommended doubling the IES research budget for K-12 education, currently about $250 million a year.

A second, complementary idea has been proposed by the National Board for Education Sciences, which oversees the research agenda of the IES. The board has recommended that Congress require federal education program grantees, as a condition of their grant awards, to participate in evaluations if asked to do so, including by random assignment where appropriate. This would help foster partnerships between program grantees and evaluators in carrying out rigorous evaluations to identify effective models and practices.

A third important step would be to create strong incentives for those who receive federal education grants to adopt research-proven models and practices in areas where such models or practices exist, and to provide assistance to put them into widespread use. A recent Education Week Commentary proposed a promising approach to advance this goal: a competitive priority for grant applicants that commit to the use of programs backed by strong evidence of effectiveness. (“Research and Effectiveness,” Oct. 18, 2006.)

A concerted effort to develop and use rigorous evidence to improve education policy, through steps such as these, would require a very modest investment of government funds in the context of the $400 billion spent each year on public K-12 education. Doubling the Institute of Education Sciences’ research budget, for instance, would cost, at an additional $250 million a year, only about 0.06 percent of the total K-12 expenditure, and the other two ideas above are budget-neutral. Yet they offer the opportunity, based on a compelling precedent from the field of medicine, to supply a critical missing piece needed to improve U.S. education: scientifically valid, actionable knowledge about what works in raising student achievement, preventing educational failure, and producing creative, motivated students who will be contributing members of society.

Related Tags:

A version of this article appeared in the May 23, 2007 edition of Education Week as Making Policy Work: The Lesson From Medicine

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Artificial Intelligence Webinar
Teaching Students to Use Artificial Intelligence Ethically
Ready to embrace AI in your classroom? Join our master class to learn how to use AI as a tool for learning, not a replacement.
Content provided by Solution Tree
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Empowering Students Using Computational Thinking Skills
Empower your students with computational thinking. Learn how to integrate these skills into your teaching and boost student engagement.
Content provided by Project Lead The Way
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
The Reality of Change: How Embracing and Planning for Change Can Shape Your Edtech Strategy
Promethean edtech experts delve into the reality of tech change and explore how embracing and planning for it can be your most powerful strategy for maximizing ROI.
Content provided by Promethean

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Superintendents Wrapped: The Songs District Leaders Listened to This Year
Five brave superintendents shared their top songs and artists from the past year with Education Week.
1 min read
A bright blue and pink background with a hand holding a phone with the spotify logo. A pair of headphones frames the cellphone.
Collage by Gina Tomko/Education Week and Canva
School & District Management Opinion I Invited Students to Help Hire a New Assistant Principal. Here’s What Happened
What began as an opportunity for the students turned into a gift for our administrative team.
3 min read
Centering students in the school community.
Vanessa Solis/Education Week via Canva
School & District Management Opinion 5 Education Leadership Lessons From Chef Ina Garten
"Less is more," "quality is everything," and more tips inspired by the art of cooking to build trust, connection in your school community.
4 min read
Screen Shot 2024 11 29 at 1.11.40 PM
Canva
School & District Management The 3 Reasons Why Students Lose the Most Instructional Time
States vary widely in how much instructional time they require, creating disparities that are only exacerbated by common interruptions.
6 min read
Photo of classroom clock.
Design Pics / Getty Images Plus