Accountability Studies Find Mixed Impact on Achievement

By Lynn Olson — June 19, 2002 4 min read
  • Save to favorites
  • Print

How effective is accountability in raising student achievement? The evidence is mixed, according to a set of research papers presented here last week.

“Most of the evidence is unpublished at this point,” noted Eric A. Hanushek, a senior fellow at the Hoover Institution of Stanford University, and the answers that exist are “partial” at best.

The dozen papers were presented at a conference called “Taking Account of Accountability: Assessing Politics and Policy,” sponsored by the program on education policy and governance and the Taubman Center for State and Local Government, both located at the John F. Kennedy School of Government at Harvard University.

A review of the existing research, by Mr. Hanushek and Margaret E. Raymond, a research fellow at the Hoover Institution, found that schools do respond to accountability systems, but not always in the ways intended. For instance, several studies indicate that the number of students excluded from testing, particularly special education students, tends to rise as new accountability systems are introduced.

Read the papers from the “Taking Account of Accountability: Assessing Politics and Policy” conference, from the John F. Kennedy School of Government’s program on education policy and governance. (Requires Adobe’s Acrobat Reader.)

At the same time, an analysis by the two researchers suggests that state accountability systems may boost student learning. They found that between 1992 and 2000, states with accountability systems experienced, on average, significantly higher growth between 4th and 8th grade on math scores on the National Assessment of Educational Progress than did states without such systems. Based on those results, Mr. Hanushek said, it appears accountability “really matters.”

A study by Thomas S. Dee, an assistant professor of economics at Swarthmore College in Pennsylvania, similarly found what he called “an interesting set of mixed results.”

Mr. Dee used data from individuals who completed the long form of the 1990 U.S. Census to examine the effects of two high school graduation requirements—minimum- competency tests and the completion of a core set of academic courses—on people who were age 18 between 1980 and 1988 and born in the United States.

He found that the introduction of such standards significantly reduced the likelihood that black males, in particular, would graduate from high school, but increased the likelihood that they would be employed.

Meanwhile, two papers on test- based accountability in Chicago drew opposite conclusions.

Anthony S. Bryk, a professor of urban education at the University of Chicago, examined elementary students’ learning gains on the Iowa Tests of Basic Skills from 1994 through 2001. He found that during the first half of the 1990s, when Chicago was focused on decentralizing the school district, students gained, on average, nearly a year of learning in reading and mathematics for every year of instruction. But as the high-stakes accountability initiatives rolled out— including mandatory summer school and possible grade retention for low achievers—the learning gains flattened.

Dueling Results

In contrast, Brian A. Jacob, an assistant professor of public policy at the Kennedy School of Government, focused on ITBS scores for students who took the tests in 1994, before the accountability measures were introduced, and in 1998, after the changes were in place. He found that students scored significantly higher on the ITBS following the introduction of high-stakes testing than their peers did prior to the new accountability policies.

While gains in math scores came disproportionately in the areas of computation and number concepts, or basic skills, the improvements in reading were spread relatively evenly across item types. Moreover, students made the largest gains on items of moderate difficulty, suggesting that teachers were not simply teaching easily mastered skills.

Dale Ballou, an associate professor of economics at the University of Massachusetts at Amherst, said methodological differences could account for the two studies’ dueling conclusions.

But Ronald Ferguson, a lecturer in public policy at the Kennedy school, suggested the biggest difference may be in how the researchers interpreted the jump in test scores in 1996-97.

Mr. Bryk attributed those gains to changes in the first wave of Chicago school reforms. Mr. Jacob attributed them to the newer wave of accountability provisions. Both could be right, Mr. Ferguson suggested, if the jump in scores stemmed from the threat of accountability rather than the actual implementation of the accountability policies, which occurred shortly afterward.

Another study, by Julian R. Betts, a professor of economics at the University of California, San Diego, and Anne Danenberg, a research associate at the Public Policy Institute of California, based in San Francisco, examined the effectiveness of California’s intervention program for low-performing schools. The Immediate Intervention/Underperforming Schools Program provides schools in the bottom half of the state’s accountability index that have not met their improvement targets with additional money and assistance if they apply for and are randomly selected into the program.

The researchers found that schools with the lowest test scores were the most likely to apply for the program, “which I think should be very heartening to anybody in favor of accountability systems,” said Mr. Betts.

Moreover, schools that applied for and were selected into the program made substantially greater achievement gains between 1999 and 2001 than schools that applied but were not picked.

But the researchers found no evidence that schools participating in the program outpaced the total pool of eligible schools, which includes those that did not apply.

“Is the program working?” asked Mr. Betts. “It’s really too early to know for sure.”

A version of this article appeared in the June 19, 2002 edition of Education Week as Accountability Studies Find Mixed Impact on Achievement


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Achievement Webinar
How To Tackle The Biggest Hurdles To Effective Tutoring
Learn how districts overcome the three biggest challenges to implementing high-impact tutoring with fidelity: time, talent, and funding.
Content provided by Saga Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Reframing Behavior: Neuroscience-Based Practices for Positive Support
Reframing Behavior helps teachers see the “why” of behavior through a neuroscience lens and provides practices that fit into a school day.
Content provided by Crisis Prevention Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Math for All: Strategies for Inclusive Instruction and Student Success
Looking for ways to make math matter for all your students? Gain strategies that help them make the connection as well as the grade.
Content provided by NMSI

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Opinion What’s Wrong With Online Credit Recovery? This Teacher Will Tell You
The “whatever it takes” approach to increasing graduation rates ends up deflating the value of a diploma.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Why a Judge Stopped Texas from Issuing A-F School Ratings
Districts argued the new metric would make it appear as if schools have worsened—even though outcomes have actually improved in many cases.
2 min read
Laura BakerEducation Week via Canva  (1)
Accountability Why These Districts Are Suing to Stop Release of A-F School Ratings
A change in how schools will be graded has prompted legal action from about a dozen school districts in Texas.
4 min read
Handwritten red letter grades cover a blue illustration of a classic brick school building.
Laura Baker, Canva
Accountability What the Research Says What Should Schools Do to Build on 20 Years of NCLB Data?
The education law yielded a cornucopia of student information, but not scalable turnaround for schools, an analysis finds.
3 min read
Photo of magnifying glass and charts.
iStock / Getty Images Plus