How effective is accountability in raising student achievement? The evidence is mixed, according to a set of research papers presented here last week.
“Most of the evidence is unpublished at this point,” noted Eric A. Hanushek, a senior fellow at the Hoover Institution of Stanford University, and the answers that exist are “partial” at best.
The dozen papers were presented at a conference called “Taking Account of Accountability: Assessing Politics and Policy,” sponsored by the program on education policy and governance and the Taubman Center for State and Local Government, both located at the John F. Kennedy School of Government at Harvard University.
A review of the existing research, by Mr. Hanushek and Margaret E. Raymond, a research fellow at the Hoover Institution, found that schools do respond to accountability systems, but not always in the ways intended. For instance, several studies indicate that the number of students excluded from testing, particularly special education students, tends to rise as new accountability systems are introduced.
Read the papers from the “Taking Account of Accountability: Assessing Politics and Policy” conference, from the John F. Kennedy School of Government’s program on education policy and governance. (Requires Adobe’s Acrobat Reader.)
At the same time, an analysis by the two researchers suggests that state accountability systems may boost student learning. They found that between 1992 and 2000, states with accountability systems experienced, on average, significantly higher growth between 4th and 8th grade on math scores on the National Assessment of Educational Progress than did states without such systems. Based on those results, Mr. Hanushek said, it appears accountability “really matters.”
A study by Thomas S. Dee, an assistant professor of economics at Swarthmore College in Pennsylvania, similarly found what he called “an interesting set of mixed results.”
Mr. Dee used data from individuals who completed the long form of the 1990 U.S. Census to examine the effects of two high school graduation requirements—minimum- competency tests and the completion of a core set of academic courses—on people who were age 18 between 1980 and 1988 and born in the United States.
He found that the introduction of such standards significantly reduced the likelihood that black males, in particular, would graduate from high school, but increased the likelihood that they would be employed.
Meanwhile, two papers on test- based accountability in Chicago drew opposite conclusions.
Anthony S. Bryk, a professor of urban education at the University of Chicago, examined elementary students’ learning gains on the Iowa Tests of Basic Skills from 1994 through 2001. He found that during the first half of the 1990s, when Chicago was focused on decentralizing the school district, students gained, on average, nearly a year of learning in reading and mathematics for every year of instruction. But as the high-stakes accountability initiatives rolled out— including mandatory summer school and possible grade retention for low achievers—the learning gains flattened.
Dueling Results
In contrast, Brian A. Jacob, an assistant professor of public policy at the Kennedy School of Government, focused on ITBS scores for students who took the tests in 1994, before the accountability measures were introduced, and in 1998, after the changes were in place. He found that students scored significantly higher on the ITBS following the introduction of high-stakes testing than their peers did prior to the new accountability policies.
While gains in math scores came disproportionately in the areas of computation and number concepts, or basic skills, the improvements in reading were spread relatively evenly across item types. Moreover, students made the largest gains on items of moderate difficulty, suggesting that teachers were not simply teaching easily mastered skills.
Dale Ballou, an associate professor of economics at the University of Massachusetts at Amherst, said methodological differences could account for the two studies’ dueling conclusions.
But Ronald Ferguson, a lecturer in public policy at the Kennedy school, suggested the biggest difference may be in how the researchers interpreted the jump in test scores in 1996-97.
Mr. Bryk attributed those gains to changes in the first wave of Chicago school reforms. Mr. Jacob attributed them to the newer wave of accountability provisions. Both could be right, Mr. Ferguson suggested, if the jump in scores stemmed from the threat of accountability rather than the actual implementation of the accountability policies, which occurred shortly afterward.
Another study, by Julian R. Betts, a professor of economics at the University of California, San Diego, and Anne Danenberg, a research associate at the Public Policy Institute of California, based in San Francisco, examined the effectiveness of California’s intervention program for low-performing schools. The Immediate Intervention/Underperforming Schools Program provides schools in the bottom half of the state’s accountability index that have not met their improvement targets with additional money and assistance if they apply for and are randomly selected into the program.
The researchers found that schools with the lowest test scores were the most likely to apply for the program, “which I think should be very heartening to anybody in favor of accountability systems,” said Mr. Betts.
Moreover, schools that applied for and were selected into the program made substantially greater achievement gains between 1999 and 2001 than schools that applied but were not picked.
But the researchers found no evidence that schools participating in the program outpaced the total pool of eligible schools, which includes those that did not apply.
“Is the program working?” asked Mr. Betts. “It’s really too early to know for sure.”
