Accountability Studies Find Mixed Impact on Achievement

By Lynn Olson — June 19, 2002 4 min read
  • Save to favorites
  • Print

How effective is accountability in raising student achievement? The evidence is mixed, according to a set of research papers presented here last week.

“Most of the evidence is unpublished at this point,” noted Eric A. Hanushek, a senior fellow at the Hoover Institution of Stanford University, and the answers that exist are “partial” at best.

The dozen papers were presented at a conference called “Taking Account of Accountability: Assessing Politics and Policy,” sponsored by the program on education policy and governance and the Taubman Center for State and Local Government, both located at the John F. Kennedy School of Government at Harvard University.

A review of the existing research, by Mr. Hanushek and Margaret E. Raymond, a research fellow at the Hoover Institution, found that schools do respond to accountability systems, but not always in the ways intended. For instance, several studies indicate that the number of students excluded from testing, particularly special education students, tends to rise as new accountability systems are introduced.

Read the papers from the “Taking Account of Accountability: Assessing Politics and Policy” conference, from the John F. Kennedy School of Government’s program on education policy and governance. (Requires Adobe’s Acrobat Reader.)

At the same time, an analysis by the two researchers suggests that state accountability systems may boost student learning. They found that between 1992 and 2000, states with accountability systems experienced, on average, significantly higher growth between 4th and 8th grade on math scores on the National Assessment of Educational Progress than did states without such systems. Based on those results, Mr. Hanushek said, it appears accountability “really matters.”

A study by Thomas S. Dee, an assistant professor of economics at Swarthmore College in Pennsylvania, similarly found what he called “an interesting set of mixed results.”

Mr. Dee used data from individuals who completed the long form of the 1990 U.S. Census to examine the effects of two high school graduation requirements—minimum- competency tests and the completion of a core set of academic courses—on people who were age 18 between 1980 and 1988 and born in the United States.

He found that the introduction of such standards significantly reduced the likelihood that black males, in particular, would graduate from high school, but increased the likelihood that they would be employed.

Meanwhile, two papers on test- based accountability in Chicago drew opposite conclusions.

Anthony S. Bryk, a professor of urban education at the University of Chicago, examined elementary students’ learning gains on the Iowa Tests of Basic Skills from 1994 through 2001. He found that during the first half of the 1990s, when Chicago was focused on decentralizing the school district, students gained, on average, nearly a year of learning in reading and mathematics for every year of instruction. But as the high-stakes accountability initiatives rolled out— including mandatory summer school and possible grade retention for low achievers—the learning gains flattened.

Dueling Results

In contrast, Brian A. Jacob, an assistant professor of public policy at the Kennedy School of Government, focused on ITBS scores for students who took the tests in 1994, before the accountability measures were introduced, and in 1998, after the changes were in place. He found that students scored significantly higher on the ITBS following the introduction of high-stakes testing than their peers did prior to the new accountability policies.

While gains in math scores came disproportionately in the areas of computation and number concepts, or basic skills, the improvements in reading were spread relatively evenly across item types. Moreover, students made the largest gains on items of moderate difficulty, suggesting that teachers were not simply teaching easily mastered skills.

Dale Ballou, an associate professor of economics at the University of Massachusetts at Amherst, said methodological differences could account for the two studies’ dueling conclusions.

But Ronald Ferguson, a lecturer in public policy at the Kennedy school, suggested the biggest difference may be in how the researchers interpreted the jump in test scores in 1996-97.

Mr. Bryk attributed those gains to changes in the first wave of Chicago school reforms. Mr. Jacob attributed them to the newer wave of accountability provisions. Both could be right, Mr. Ferguson suggested, if the jump in scores stemmed from the threat of accountability rather than the actual implementation of the accountability policies, which occurred shortly afterward.

Another study, by Julian R. Betts, a professor of economics at the University of California, San Diego, and Anne Danenberg, a research associate at the Public Policy Institute of California, based in San Francisco, examined the effectiveness of California’s intervention program for low-performing schools. The Immediate Intervention/Underperforming Schools Program provides schools in the bottom half of the state’s accountability index that have not met their improvement targets with additional money and assistance if they apply for and are randomly selected into the program.

The researchers found that schools with the lowest test scores were the most likely to apply for the program, “which I think should be very heartening to anybody in favor of accountability systems,” said Mr. Betts.

Moreover, schools that applied for and were selected into the program made substantially greater achievement gains between 1999 and 2001 than schools that applied but were not picked.

But the researchers found no evidence that schools participating in the program outpaced the total pool of eligible schools, which includes those that did not apply.

“Is the program working?” asked Mr. Betts. “It’s really too early to know for sure.”

A version of this article appeared in the June 19, 2002 edition of Education Week as Accountability Studies Find Mixed Impact on Achievement

Commenting has been disabled on effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Ensuring Continuity of Learning: How to Prepare for the Next Disruption
Across the country, K-12 schools and districts are, again, considering how to ensure effective continuity of learning in the face of emerging COVID variants, politicized debates, and more. Learn from Alexandria City Public Schools superintendent
Content provided by Class
Teaching Profession Live Online Discussion What Have We Learned From Teachers During the Pandemic?
University of California, Santa Cruz, researcher Lora Bartlett and her colleagues spent months studying how the pandemic affected classroom teachers. We will discuss the takeaways from her research not only for teachers, but also for

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Did Washington D.C.'s Education Overhaul Help Black Children? This Study Says Yes
Researchers said the district's "market-based" reforms accelerated achievement versus other districts and states.
5 min read
Accountability Opinion What Next-Gen Accountability Can Learn From No Child Left Behind
As we ponder what's next for accountability and assessment, we’d benefit from checking the rearview mirror more attentively and more often.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Opinion Let’s Make Transparency the Pandemic’s Educational Legacy
Transparency can strengthen school communities, allow parents to see what’s happening, and provide students more of the support they need.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability The Feds Offered Waivers on ESSA Accountability. Here's Where States Stand on Getting Them
While they get less attention than testing waivers, flexibility related to low-performing schools is an important federal and state issue.
5 min read
Image of a student taking a test with a mask on.
Rich Vintage/E+