Special Report

‘Race to Top’ Said to Lack Key Science

By Debra Viadero — October 02, 2009 8 min read

Among education researchers, one complaint about the U.S. Department of Education under former President George W. Bush was that it relentlessly promoted “scientific research in education,” while at the same time endorsing some policies that lacked solid research evidence.

With recently published draft guidelines for federal economic-stimulus money and Title I aid, critics are beginning to ask whether much has changed under the Obama administration.

“What is extraordinary about these regulations is that they have no credible basis in research. They just happen to be the programs and approaches favored by the people in power,” writes Diane Ravitch, a research professor of education at New York University, in her blog, Bridging Differences, which is hosted by edweek.org. She served as the Education Department’s assistant secretary for educational research and improvement under President George H.W. Bush.

For their part, department officials are not yet answering the criticism. They did not respond to repeated requests from Education Week to address such complaints.

But some outside experts suggested the scant attention to research in the department’s draft guidelines may be more reflective of a general lack of credible findings from education research, which has long been considered underfunded compared with “hard” sciences, such as medicine.

“There isn’t a whole lot of conclusive findings about different strategies for reform, so policymakers have to look at what is the best available evidence, even though the evidence might not be rigorous or powerful,” said James W. Kohlmoos, the president of the Knowledge Alliance, a Washington-based group that represents research organizations. “They’re still working off evidence that’s not fully developed, and it’s not their fault. It’s a generation’s fault for not paying attention to this.”

The lack of research reflected in some of the department’s school-improvement initiatives is a disappointment, though, to researchers and advocates who were encouraged by President Barack Obama’s Inauguration Day pledge to “give science its rightful place” in government decisionmaking.

An ‘Opposite Approach’

Both the Bush administration and President Obama “have supported using data to determine ‘what works,’ ” economists Sean P. Corcoran and Joydeep Roy of the Washington-based Economic Policy Institute write in a letter filed in response to the proposed guidelines for the $4 billion Race to the Top fund, part of the economic-stimulus legislation. Yet, in some areas, they say, the proposed rules “take the exact opposite approach.”

In particular, Ms. Ravitch, Mr. Corcoran, Mr. Roy, and scholars such as Duke University’s Helen F. Ladd oppose two priorities at the heart of the program that they say lack research evidence: evaluating teachers based on students’ standardized test scores and promoting the growth of charter schools.

“One theory of action seems to be that holding teachers more accountable for the gain in their students’ test scores will induce them to become better teachers,” writes Ms. Ladd, a professor of public-policy studies and economics. “At this point, I am not aware of any credible evidence in support of that proposition.”

Another expert, Matthew G. Springer, the director of the National Center on Performance Incentives at Vanderbilt University in Nashville, Tenn., confirmed Ms. Ladd’s observation. In a 2008 research review he co-wrote on the use of merit-pay programs in education, he found only eight studies on the topic, the most rigorous of which were conducted outside the United States. Some studies yielded positive results; others pointed to possible negative consequences.

Skepticism of efforts to evaluate educators based on their students’ test scores was not universal among academics, though. “We strongly support the focus of the Race to the Top program on teacher effectiveness and achieving equity in distribution of effective teachers,” wrote eight academics in a comment letter submitted by Thomas J. Kane, a professor of education and economics at Harvard University in Cambridge, Mass., and the deputy director of research and data for the Seattle-based Bill & Melinda Gates Foundation’s education program. The scholars recommended, however, adding some more specifics to the proposed rules.

Thin and Thinner

The evidence behind Race to the Top’s call for giving priority to states that don’t impose caps on the growth of charter schools, likewise, is far from definitive, various commenters said.

Recent studies in Boston and New York City have found that charter school students outperform regular public school peers who applied to charter schools but failed to win a seat. But a study of charters in 15 states and the District of Columbia by the Center for Research on Education Outcomes at Stanford University found that students in more than 80 percent of the charter schools performed the same as, or worse than, students in regular public schools on mathematics tests. Achievement from school to school was highly variable, the study found. (“N.Y.C. Study Finds Gains for Charters,” Sept. 30, 2009.)

The American Educational Research Association, a Washington-based group that represents 25,000 education researchers, notes in its Race to the Top comments that “there has been much less research about the turnaround strategies identified in the proposed regulations than about charter schools, and, consequently, even less is known about conditions required for their success.”

Yet the Education Department, in both the proposed Race to the Top rules and in draft regulations governing the spending of $3.5 billion in new school-improvement aid for the nation’s 5,000 worst-performing schools, spells out specific interventions states should use to qualify for the funding. They range from implementing a mandatory basket of “transformation” strategies to closing the school and sending students to higher-achieving schools to “restarting” the school by turning it over to a charter- or education-management organization.

Evidence is similarly meager for the emphasis in the proposed Race to the Top guidelines on developing common academic standards, said Grover J. “Russ” Whitehurst,the former director of the Institute of Education Sciences, the main research arm for the department.

“In general, I would give the administration a B+ for sensitivity to research in formulating Race to the Top guidelines and other initiatives,” wrote Mr. Whitehurst, now the director of the Brown Center for Education at the Brookings Institution in Washington. “I would give them an A in the future if they would only acknowledge when policy is in front of evidence—sometimes it must be.”

Little-Used Evidence

Indeed, a report released last month shows, policymakers at all levels of the education system—local, state, and federal—often do not rely on research to make policy decisions. They respond more often to trusted colleagues, political pressure, information from professional groups, constituents, and other sources.

“It was a common perception of the study participants that research could be shaped to say anything, that one piece of research often conflicts with another, and much research is not timely for users’ needs,” concludes the report by Education Northwest, a Portland, Ore., research group that was formerly known as Northwest Regional Education Laboratory. The findings draw on conversations with six focus groups made up of education leaders from varying levels of government.

Mr.Whitehurst said the department should get “a much lower grade on their plans to evaluate and learn from the huge investments they are making” with economic-stimulus dollars. The proposed regulations for turning around failing schools and the Race to the Top program make little, or no, mention of any evaluation plans.

However, John Q. Easton, IES’s director, said in an interview last week that his agency is developing plans for a massive, multilayered, cross-cutting evaluation of states’ stimulus-funded efforts that could be financed in part with IES funds.

“We’re going to start out by looking at what’s being done with the funds, what kinds of activities states are undertaking, and then create evaluations around the strategies rather than around the programs themselves,” he said.

Department officials have also said soon-to-be proposed rules for the economic-stimulus program’s $650 million Investing in Innovtion grant program, called the “i3 Fund,” set aside a prominent role for research. They will reserve the largest grants for schools, districts, and nonprofit organizations that want to finance programs that have proven track records and are ready to expand.

But, if federal officials are going to require the use of research-based programs, they are also going to have to define what that means, experts said. While the phrase “scientifically based research” has fallen out of favor with the advent of the Obama administration, federal education officials have yet to replace it with a different definition. Both the AERA and the Knowledge Alliance, in their comments on Race to the Top, urged Mr. Duncan to include suggested definitions of “scientifically valid research” in final program guidelines later this fall.

But, as some experts pointed out, with final regulations yet to come on any of the new economic-stimulus programs, any judgments on the department’s stance with regard to research may be premature.

“I don’t think it necessarily precludes them from being more reflective down the road,” said Gerald Sroufe, the AERA’s director of government relations. “Look at Afghanistan.”

Coverage of the American Recovery and Reinvestment Act is supported in part by grants from the William and Flora Hewlett Foundation, at www.hewlett.org, and the Charles Stewart Mott Foundation, at www.mott.org.
A version of this article appeared in the October 07, 2009 edition of Education Week as ‘Race to Top’ Said to Lack Key Science


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Professional Development Webinar
Building Leadership Excellence Through Instructional Coaching
Join this webinar for a discussion on instructional coaching and ways you can link your implement or build on your program.
Content provided by Whetstone Education/SchoolMint
Teaching Webinar Tips for Better Hybrid Learning: Ask the Experts What Works
Register and ask your questions about hybrid learning to our expert panel.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Families & the Community Webinar
Family Engagement for Student Success With Dr. Karen Mapp
Register for this free webinar to learn how to empower and engage families for student success featuring Karen L. Mapp.
Content provided by Panorama Education & PowerMyLearning

EdWeek Top School Jobs

[2021-2022] Founding Middle School Academic Dean
New York, NY, US
DREAM Charter School
Hiring Bilingual and Special Education Teachers NOW!
Newark, New Jersey
Newark Public Schools
DevOps Engineer
Portland, OR, US
Northwest Evaluation Association
User Experience Analyst
Portland, OR, US
Northwest Evaluation Association

Read Next

Federal Biden Announces Goal to Get Educators the COVID-19 Vaccine This Month
President Joe Biden pushes states to get educators at least one dose by the end of March to help schools resume in-person learning.
4 min read
John Battle High School teacher Jennifer Daniel receives her COVID-19 vaccine on Jan. 11, 2021. Teachers received their first vaccine during an all-day event at the Virginia Highlands Higher Education Center in Abingdon, Va.
John Battle High School teacher Jennifer Daniel receives her COVID-19 vaccine on Jan. 11, 2021. Teachers received their first vaccine during an all-day event at the Virginia Highlands Higher Education Center in Abingdon, Va.
David Crigger/Bristol Herald Courier via AP
Federal Explainer Miguel Cardona, U.S. Secretary of Education: Background and Achievements
Background and highlights of Miguel Cardona's tenure as the twelfth U.S. Secretary of Education.
Education Week Library
2 min read
Miguel Cardona, President-elect Joe Biden's nominee for Secretary of Education, speaks after being introduced at The Queen Theater in Wilmington, Del., on Dec. 23, 2020.
Miguel Cardona, U.S. Secretary of Education, speaks after being put forward for the position by then-President-elect Joe Biden in December 2020.
Carolyn Kaster/AP
Federal Senate Confirms Miguel Cardona as Education Secretary
The former Connecticut education commissioner got his start as an elementary school teacher and was a principal and school administrator.
2 min read
Miguel Cardona, President-elect Joe Biden's nominee for Secretary of Education, speaks after being introduced at The Queen Theater in Wilmington, Del., Wednesday, Dec. 23, 2020.
Miguel Cardona was confirmed by the Senate to serve as U.S. Secretary of Education. The former Connecticut education commissioner has worked as a teacher, principal, and district administrator.
Carolyn Kaster/AP
Federal Biden Legal Team Steps Back From Trump Stance on Transgender Female Sports Participation
The Education Department's office for civil rights pulls a letter that said Connecticut's transgender-inclusive policy violates Title IX.
4 min read
Bloomfield High School transgender athlete Terry Miller, second from left, wins the final of the 55-meter dash over transgender athlete Andraya Yearwood, far left, and other runners in the Connecticut girls Class S indoor track meet at Hillhouse High School in New Haven, Conn on Feb. 7, 2019. Transgender athletes are getting an ally in the White House next week as they seek to participate as their identified gender in high school and college sports. Attorneys on both sides say they expect President-elect Joe Biden’s Department of Education will switch sides in legal battles that could go a long way in determining whether transgender athletes are treated by the sex on their birth certificates or by how they identify.
Bloomfield High School transgender athlete Terry Miller, second from left, wins over transgender athlete Andraya Yearwood, far left, and other runners in an event in New Haven, Conn. The two transgender athletes are at the center of a legal fight in Connecticut over the participation of transgender female athletes in girls' or women's sports.
Pat Eaton-Robb/AP