Accountability

Accountability Studies Find Mixed Impact on Achievement

By Lynn Olson — June 19, 2002 4 min read
  • Save to favorites
  • Print

How effective is accountability in raising student achievement? The evidence is mixed, according to a set of research papers presented here last week.

“Most of the evidence is unpublished at this point,” noted Eric A. Hanushek, a senior fellow at the Hoover Institution of Stanford University, and the answers that exist are “partial” at best.

The dozen papers were presented at a conference called “Taking Account of Accountability: Assessing Politics and Policy,” sponsored by the program on education policy and governance and the Taubman Center for State and Local Government, both located at the John F. Kennedy School of Government at Harvard University.

A review of the existing research, by Mr. Hanushek and Margaret E. Raymond, a research fellow at the Hoover Institution, found that schools do respond to accountability systems, but not always in the ways intended. For instance, several studies indicate that the number of students excluded from testing, particularly special education students, tends to rise as new accountability systems are introduced.

Read the papers from the “Taking Account of Accountability: Assessing Politics and Policy” conference, from the John F. Kennedy School of Government’s program on education policy and governance. (Requires Adobe’s Acrobat Reader.)

At the same time, an analysis by the two researchers suggests that state accountability systems may boost student learning. They found that between 1992 and 2000, states with accountability systems experienced, on average, significantly higher growth between 4th and 8th grade on math scores on the National Assessment of Educational Progress than did states without such systems. Based on those results, Mr. Hanushek said, it appears accountability “really matters.”

A study by Thomas S. Dee, an assistant professor of economics at Swarthmore College in Pennsylvania, similarly found what he called “an interesting set of mixed results.”

Mr. Dee used data from individuals who completed the long form of the 1990 U.S. Census to examine the effects of two high school graduation requirements—minimum- competency tests and the completion of a core set of academic courses—on people who were age 18 between 1980 and 1988 and born in the United States.

He found that the introduction of such standards significantly reduced the likelihood that black males, in particular, would graduate from high school, but increased the likelihood that they would be employed.

Meanwhile, two papers on test- based accountability in Chicago drew opposite conclusions.

Anthony S. Bryk, a professor of urban education at the University of Chicago, examined elementary students’ learning gains on the Iowa Tests of Basic Skills from 1994 through 2001. He found that during the first half of the 1990s, when Chicago was focused on decentralizing the school district, students gained, on average, nearly a year of learning in reading and mathematics for every year of instruction. But as the high-stakes accountability initiatives rolled out— including mandatory summer school and possible grade retention for low achievers—the learning gains flattened.

Dueling Results

In contrast, Brian A. Jacob, an assistant professor of public policy at the Kennedy School of Government, focused on ITBS scores for students who took the tests in 1994, before the accountability measures were introduced, and in 1998, after the changes were in place. He found that students scored significantly higher on the ITBS following the introduction of high-stakes testing than their peers did prior to the new accountability policies.

While gains in math scores came disproportionately in the areas of computation and number concepts, or basic skills, the improvements in reading were spread relatively evenly across item types. Moreover, students made the largest gains on items of moderate difficulty, suggesting that teachers were not simply teaching easily mastered skills.

Dale Ballou, an associate professor of economics at the University of Massachusetts at Amherst, said methodological differences could account for the two studies’ dueling conclusions.

But Ronald Ferguson, a lecturer in public policy at the Kennedy school, suggested the biggest difference may be in how the researchers interpreted the jump in test scores in 1996-97.

Mr. Bryk attributed those gains to changes in the first wave of Chicago school reforms. Mr. Jacob attributed them to the newer wave of accountability provisions. Both could be right, Mr. Ferguson suggested, if the jump in scores stemmed from the threat of accountability rather than the actual implementation of the accountability policies, which occurred shortly afterward.

Another study, by Julian R. Betts, a professor of economics at the University of California, San Diego, and Anne Danenberg, a research associate at the Public Policy Institute of California, based in San Francisco, examined the effectiveness of California’s intervention program for low-performing schools. The Immediate Intervention/Underperforming Schools Program provides schools in the bottom half of the state’s accountability index that have not met their improvement targets with additional money and assistance if they apply for and are randomly selected into the program.

The researchers found that schools with the lowest test scores were the most likely to apply for the program, “which I think should be very heartening to anybody in favor of accountability systems,” said Mr. Betts.

Moreover, schools that applied for and were selected into the program made substantially greater achievement gains between 1999 and 2001 than schools that applied but were not picked.

But the researchers found no evidence that schools participating in the program outpaced the total pool of eligible schools, which includes those that did not apply.

“Is the program working?” asked Mr. Betts. “It’s really too early to know for sure.”

A version of this article appeared in the June 19, 2002 edition of Education Week as Accountability Studies Find Mixed Impact on Achievement

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
(Re)Focus on Dyslexia: Moving Beyond Diagnosis & Toward Transformation
Move beyond dyslexia diagnoses & focus on effective literacy instruction for ALL students. Join us to learn research-based strategies that benefit learners in PreK-8.
Content provided by EPS Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Cohesive Instruction, Connected Schools: Scale Excellence District-Wide with the Right Technology
Ensure all students receive high-quality instruction with a cohesive educational framework. Learn how to empower teachers and leverage technology.
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School Climate & Safety Webinar
How to Use Data to Combat Bullying and Enhance School Safety
Join our webinar to learn how data can help identify bullying, implement effective interventions, & foster student well-being.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability What the Research Says More than 1 in 4 Schools Targeted for Improvement, Survey Finds
The new federal findings show schools also continue to struggle with absenteeism.
2 min read
Vector illustration of diverse children, students climbing up on a top of a stack of staggered books.
iStock/Getty
Accountability Opinion What’s Wrong With Online Credit Recovery? This Teacher Will Tell You
The “whatever it takes” approach to increasing graduation rates ends up deflating the value of a diploma.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Why a Judge Stopped Texas from Issuing A-F School Ratings
Districts argued the new metric would make it appear as if schools have worsened—even though outcomes have actually improved in many cases.
2 min read
Laura BakerEducation Week via Canva  (1)
Canva
Accountability Why These Districts Are Suing to Stop Release of A-F School Ratings
A change in how schools will be graded has prompted legal action from about a dozen school districts in Texas.
4 min read
Handwritten red letter grades cover a blue illustration of a classic brick school building.
Laura Baker, Canva