Federal Opinion

Research and Effectiveness

By Robert E. Slavin — October 17, 2006 6 min read

—Patti Raine


Over the summer, an insightful essay in these pages contrasted two conflicting philosophies embedded within the No Child Left Behind Act: one dedicated to “what works,” the other to “whatever works.” (“What Works vs. Whatever Works,” July 26, 2006.) The “what works” philosophy, explained the author, expects the federal law to promote the use of proven programs. The “whatever works” philosophy, in contrast, seeks to set tough accountability standards but then allow great freedom for schools to do whatever it is that results in success for children. The struggle between adherents of these opposing philosophies, he argued, will determine the future course of the federal legislation.

This description by former U.S. Department of Education official Michael J. Petrilli of the two “theories of action” and their centrality to the debate over the law’s upcoming reauthorization is perceptive. But No Child Left Behind in theory does not resemble No Child Left Behind in fact. Despite the legislation’s many references to “scientifically based research,” evidence of effectiveness is not yet a significant factor in its implementation.

Early on in the “No Child” era, Education Department leaders seemed serious about the “what works” philosophy. But they faced a dilemma in putting it into practice. They felt there was too little high-quality research on too few programs, and too few reviews of research on practical programs, to make confident statements about what works in most arenas. They abandoned the idea of promoting programs with strong evidence of effectiveness and instead applied a far lower standard of evidence, promoting programs “based on scientifically-based evidence,” meaning that they somehow incorporate elements that have been successfully evaluated, even if the programs themselves have not.

Accountability alone is insufficient. American education has been pursuing accountability-only policies for a quarter-century, with little to show for it.

At the same time, the reconfigured federal research entity, the Institute of Education Sciences, under the leadership of Grover J. “Russ” Whitehurst, set to work to add to the research base for educational practice. Since 2002, the IES has funded scores of high-quality, randomized evaluations of all sorts of programs and practices, prekindergarten to grade 12. It also is funding development of many new programs, as well as basic research it hopes will ultimately lead to applied programs. The IES established the What Works Clearinghouse to review existing research, and is now talking about a second “promising practices” initiative. Meanwhile, the National Institute of Child Health and Human Development and private foundations, especially the W.T. Grant Foundation, are also funding much high-quality research and development on practical programs for schools.

All of this unprecedented R&D activity focused on directly improving educational practice is now beginning to bear fruit. As the No Child Left Behind Act heads toward reauthorization, it is time to think about how the law can capitalize on the advances in research set in motion.

Instead of being set in opposition to each other, the “what works” and “whatever works” philosophies should be seen as partners: two approaches to reform that complement each other. Some form of accountability for student achievement is necessary, and government has little reason to prescribe programs or practices to schools that are demonstrably succeeding.

But accountability alone is insufficient. American education has been pursuing accountability-only policies for a quarter-century, with little to show for it. National Assessment of Educational Progress reading scores have hardly changed since 1980, when the accountability movement got under way.

Educational practice will not change for the better until educators have and use more-effective tools, such as programs, technologies, and practices known from rigorous evaluation research to be effective. Accountability pressure should lead educators to seek effective programs and implement them with integrity. When educators feel pressure to increase scores but do not have available proven solutions, they all too often resort to gamesmanship, adopting practices that increase scores without increasing learning, such as narrowing the curriculum, holding more students back, and emphasizing test-taking skills over important learning.

The question for policymakers serious about using the No Child Left Behind law as a lever to bring about genuine change is how to balance a focus on proven programs with a need for flexibility and local control. In general, the rule should be that as long as a school is meeting performance standards and is not seeking additional federal funding, the federal government has no business telling local educators what to do. But when schools are not meeting standards, and especially when they are seeking discretionary funding, the federal government has the right and the responsibility to encourage the use of programs known to be effective.

One approach that might produce the proper balance would be to include in a broad category of federal requests for applications for schools, districts, and states a bonus of up to 10 points, on a scale of 100, if the applicant commits to the use of programs with strong evidence of effectiveness. “Strong evidence of effectiveness” means that the program has been evaluated in comparison to randomized or matched control groups in studies that, for example, meet the standards of the What Works Clearinghouse. If the program area had not yet been reviewed by the clearinghouse, applicants could present evidence from other reviews using similar standards. If applicants were participating in or carrying out bona fide randomized evaluations of promising programs not yet known to be effective, they might also qualify for the bonus.

When educators feel pressure to increase scores but do not have available proven solutions, they all too often resort to gamesmanship.

Proposers would receive the bonus only if they committed to fully implementing whatever research-proven program they selected, with adequate professional development and follow-up. The bonus would be available for discretionary programs, such as Reading First, Striving Readers, and others intended to increase student achievement. Schools not meeting standards might be given 10 bonus points if their applications for additional funding to help them achieve their goals promised to implement proven programs. In fact, the same principle might be applied to funding in any area in which there are programs with strong evidence of effectiveness.

The effect of such a bonus would be dramatic. No grant writer lightly ignores the possibility of 10 points. Yet the offer of the bonus would not force anyone to do anything.

Moreover, the 10 percent bonus could have a substantial effect on the entire R&D enterprise, as developers, publishers, and technology companies would have a strong incentive to carry out or commission high-quality research on promising programs. Government, especially at the federal level, would also have a strong incentive to fund R&D on promising programs, as policymakers would know that successfully evaluated programs were likely to be broadly disseminated.

Any successful initiative to promote the use of proven programs could finally move the practice of education toward the kind of progress over time that medicine, agriculture, and other fields have experienced when government action has broken through resistance to innovation. Such an initiative could unleash the creative power of American innovators, who could find effective solutions to educational problems if they were certain that these solutions would be widely adopted. This approach is essentially cost-free, as it would simply put to better use funds that federal, state, and local governments already are spending on programs for struggling schools.

America can solve its educational problems the way it has solved so many others—through innovation, creativity, and research and development. It is time for government to take the actions that fit the words in the No Child Left Behind Act promoting the use of proven programs.

A version of this article appeared in the October 18, 2006 edition of Education Week as Research and Effectiveness


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Here to Stay – Pandemic Lessons for EdTech in Future Development
What technology is needed in a post pandemic district? Learn how changes in education will impact development of new technologies.
Content provided by AWS
School & District Management Live Online Discussion A Seat at the Table: Strategies & Tips for Complex Decision-Making
Schools are working through the most disruptive period in the history of modern education, facing a pandemic, economic problems, social justice issues, and rapid technological change all at once. But even after the pandemic ends,

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Biden Pick for Education Civil Rights Office Has History With Racial Equity, LGBTQ Issues
Biden selected Catherine Lhamon to lead the Education Department's civil rights work, a role she also held in the Obama administration.
2 min read
Flags decorate a space outside the office of the Education Secretary at the Education Department in Washington on Aug. 9, 2017.
Flags decorate a space outside the office of the Education Secretary at the Education Department in Washington on Aug. 9, 2017.
Jacquelyn Martin/AP
Federal Lawmakers Press CDC About Teachers' Union Influence on School Reopening Guidance
Republican senators asked CDC Director Rochelle Walensky about reports a teachers' union had input on guidance for schools on COVID-19.
3 min read
Dr. Rochelle Walensky speaks during an event in Wilmington, Del., to announce President-elect Joe Biden's health care team on Dec. 8, 2020.
Dr. Rochelle Walensky, CDC director, speaks during an event in Wilmington, Del., to announce then-President-elect Joe Biden's health care team on Dec. 8, 2020.
Susan Walsh/AP
Federal Biden Taps Ex-Obama Aide Roberto Rodriguez for Key Education Department Job
Rodriguez served as a top education staffer to President Barack Obama and currently leads a teacher-advocacy organization.
3 min read
Federal Biden Pitches Plan to Expand Universal Pre-K, Free School Meal Programs, Teacher Training
The president's $1.8 trillion American Families Plan faces strong headwinds as Congress considers other costly administration proposals.
8 min read
President Joe Biden addresses Congress from the House chamber. Behind him are Vice President Kamala Harris and Speaker of the House Nancy Pelosi.
President Joe Biden addresses a joint session of Congress Wednesday night, as Vice President Kamala Harris, left, and House Speaker Nancy Pelosi of Calif., applaud.<br/>
Chip Somodevilla/AP