Accountability Studies Find Mixed Impact on Achievement

By Lynn Olson — June 19, 2002 4 min read
  • Save to favorites
  • Print

How effective is accountability in raising student achievement? The evidence is mixed, according to a set of research papers presented here last week.

“Most of the evidence is unpublished at this point,” noted Eric A. Hanushek, a senior fellow at the Hoover Institution of Stanford University, and the answers that exist are “partial” at best.

The dozen papers were presented at a conference called “Taking Account of Accountability: Assessing Politics and Policy,” sponsored by the program on education policy and governance and the Taubman Center for State and Local Government, both located at the John F. Kennedy School of Government at Harvard University.

A review of the existing research, by Mr. Hanushek and Margaret E. Raymond, a research fellow at the Hoover Institution, found that schools do respond to accountability systems, but not always in the ways intended. For instance, several studies indicate that the number of students excluded from testing, particularly special education students, tends to rise as new accountability systems are introduced.

Read the papers from the “Taking Account of Accountability: Assessing Politics and Policy” conference, from the John F. Kennedy School of Government’s program on education policy and governance. (Requires Adobe’s Acrobat Reader.)

At the same time, an analysis by the two researchers suggests that state accountability systems may boost student learning. They found that between 1992 and 2000, states with accountability systems experienced, on average, significantly higher growth between 4th and 8th grade on math scores on the National Assessment of Educational Progress than did states without such systems. Based on those results, Mr. Hanushek said, it appears accountability “really matters.”

A study by Thomas S. Dee, an assistant professor of economics at Swarthmore College in Pennsylvania, similarly found what he called “an interesting set of mixed results.”

Mr. Dee used data from individuals who completed the long form of the 1990 U.S. Census to examine the effects of two high school graduation requirements—minimum- competency tests and the completion of a core set of academic courses—on people who were age 18 between 1980 and 1988 and born in the United States.

He found that the introduction of such standards significantly reduced the likelihood that black males, in particular, would graduate from high school, but increased the likelihood that they would be employed.

Meanwhile, two papers on test- based accountability in Chicago drew opposite conclusions.

Anthony S. Bryk, a professor of urban education at the University of Chicago, examined elementary students’ learning gains on the Iowa Tests of Basic Skills from 1994 through 2001. He found that during the first half of the 1990s, when Chicago was focused on decentralizing the school district, students gained, on average, nearly a year of learning in reading and mathematics for every year of instruction. But as the high-stakes accountability initiatives rolled out— including mandatory summer school and possible grade retention for low achievers—the learning gains flattened.

Dueling Results

In contrast, Brian A. Jacob, an assistant professor of public policy at the Kennedy School of Government, focused on ITBS scores for students who took the tests in 1994, before the accountability measures were introduced, and in 1998, after the changes were in place. He found that students scored significantly higher on the ITBS following the introduction of high-stakes testing than their peers did prior to the new accountability policies.

While gains in math scores came disproportionately in the areas of computation and number concepts, or basic skills, the improvements in reading were spread relatively evenly across item types. Moreover, students made the largest gains on items of moderate difficulty, suggesting that teachers were not simply teaching easily mastered skills.

Dale Ballou, an associate professor of economics at the University of Massachusetts at Amherst, said methodological differences could account for the two studies’ dueling conclusions.

But Ronald Ferguson, a lecturer in public policy at the Kennedy school, suggested the biggest difference may be in how the researchers interpreted the jump in test scores in 1996-97.

Mr. Bryk attributed those gains to changes in the first wave of Chicago school reforms. Mr. Jacob attributed them to the newer wave of accountability provisions. Both could be right, Mr. Ferguson suggested, if the jump in scores stemmed from the threat of accountability rather than the actual implementation of the accountability policies, which occurred shortly afterward.

Another study, by Julian R. Betts, a professor of economics at the University of California, San Diego, and Anne Danenberg, a research associate at the Public Policy Institute of California, based in San Francisco, examined the effectiveness of California’s intervention program for low-performing schools. The Immediate Intervention/Underperforming Schools Program provides schools in the bottom half of the state’s accountability index that have not met their improvement targets with additional money and assistance if they apply for and are randomly selected into the program.

The researchers found that schools with the lowest test scores were the most likely to apply for the program, “which I think should be very heartening to anybody in favor of accountability systems,” said Mr. Betts.

Moreover, schools that applied for and were selected into the program made substantially greater achievement gains between 1999 and 2001 than schools that applied but were not picked.

But the researchers found no evidence that schools participating in the program outpaced the total pool of eligible schools, which includes those that did not apply.

“Is the program working?” asked Mr. Betts. “It’s really too early to know for sure.”

A version of this article appeared in the June 19, 2002 edition of Education Week as Accountability Studies Find Mixed Impact on Achievement


Recruitment & Retention Live Online Discussion A Seat at the Table: Chronic Teacher Shortage: Where Do We Go From Here?  
Join Peter DeWitt, Michael Fullan, and guests for expert insights into finding solutions for the teacher shortage.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Reading & Literacy Webinar
The Science of Reading: Tools to Build Reading Proficiency
The Science of Reading has taken education by storm. Learn how Dr. Miranda Blount transformed literacy instruction in her state.
Content provided by hand2mind
Student Achievement K-12 Essentials Forum Tutoring Done Right: How to Get the Highest Impact for Learning Recovery
Join us as we highlight and discuss the evidence base for tutoring, best practices, and different ways to provide it at scale.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Timeline: How Federal School Accountability Has Waxed and Waned
From its origins in the 1990s to the most-recent tack, see how the federal approach to accountability has shifted.
4 min read
President George W. Bush, left, participates in the swearing-in ceremony for the Secretary of Education Margaret Spellings, center, at the U.S. Dept. of Education on Jan. 31, 2005 in Washington. On the far right holding a bible is her husband Robert Spellings.
President George W. Bush, left, participates in the swearing-in ceremony for the Secretary of Education Margaret Spellings, center, at the U.S. Dept. of Education on Jan. 31, 2005 in Washington. On the far right holding a bible is her husband Robert Spellings.
AP Photo/Pablo Martinez Monsivais
Accountability School Accountability Is Restarting After a Two-Year Pause. Here's What That Means
For a moment, the COVID-19 pandemic succeeded in doing what periodic protests about school accountability couldn't: Halting it.
10 min read
Illustration of a gauge.
Accountability Opinion Let's Take a Holistic Approach to Judging Schools
Parents wouldn't judge their kids based on a single factor. So, says Ron Berger of EL Education, why must schools use a lone test score?
8 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Accountability Opinion Are K-12 State Tests Like a Visit to the Pediatrician?
Even if the doctor’s trip isn’t pleasant, at least parents get something out of it they believe is worthwhile.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty