School & District Management

Study Offers Mixed Results On Impact of High-Stakes Tests

By Debra Viadero — January 28, 2004 3 min read
  • Save to favorites
  • Print

Efforts in more than half the states to tie major consequences to students’ test scores are translating into academic gains, according to the latest in a series of studies on the policy approach known as high-stakes testing.

The report, “Reconsidering the Impact of High-Stakes Testing,” is available from the Education Policy Analysis Archives.

Or then again, maybe they’re not.

The study, published this month in the online journal Education Policy Analysis Archives, draws on eight years of national testing data to compare states with traditional “low stakes” testing policies against those with “high stakes” systems. Under high-stakes policies, students’ scores are used to decide which teachers or schools win cash bonuses, whether students graduate or move on to the next grade, or what schools are subject to takeover by their districts or states.

The report follows half a dozen other studies over the past year that have used similar techniques to evaluate the effects of such accountability systems. (“Study Finds Higher Gains in States With High-Stakes Tests,” April 16, 2003).

“I was intrigued by the fact that different researchers with different ideological stances were coming to different conclusions from the same data,” said Henry I. Braun, the author of the new study. “I was also motivated by a sense that the world of research is very complex and we are not, in our research worlds, respectful enough of that complexity.”

Sorting It Out

Mr. Braun, a statistician with the Princeton, N.J.-based Educational Testing Service, used four different methods to compare changes in states’ scores on National Assessment of Educational Progress mathematics tests between 1992 and 2000.

The comparisons pitted the 18 states that some previous researchers have identified as having high-stakes systems against 32 with lower-pressure accountability systems.

Looking first at overall changes in the states’ 4th and 8th grade test scores over that period, Mr. Braun, like most of his predecessors, found that students’ academic gains were greater in states, such as Texas and North Carolina, that had high- pressure testing systems.

What’s more, he said, the trend could not be explained by statistical errors or the fact that some of the states showing the biggest improvements had also been excluding growing percentages of special education students from the tests.

In the 4th grade, the difference in mean scores between the high-stakes and low-stakes states was 4.3 score points; in 8th grade, it was 3.99 score points.

The opposite occurred, though, when Mr. Braun took a look at how cohorts of students fared on the tests over time. (He compared 4th graders’ scores with the 8th grade scores in the same states four years later.)

That time around, the improvements in academic achievement were greater—albeit to a lesser degree—in the states with low-pressure testing systems.

Mr. Braun said the differing results didn’t surprise him.

“You cannot look at high-stakes testing in isolation from other things going on in the state,” he said. “Many education reforms can be assisted or thwarted by other education reforms going on at the same time.”

In an effort to take a broader look, Mr. Braun reconfigured the data to factor in a measure that rated states on their education activism. It assigned states grades based on whether they had enacted—or were about to enact—22 school improvement efforts, such as professional standards for teachers or subject-matter standards.

But he found little correlation between the level of states’ education activism and their students’ test-score changes over time.

Mr. Braun also looked at changes in scores for the bottom 25 percent of students in each of the states. Gains were greater in the states that put more pressure on students or schools for test-score improvements.

“All of this is about which states you include and which states you do not include,” said David C. Berliner, an education professor at Arizona State University in Tempe whose own study on high-stakes testing helped spark the spate of research on the subject. (“Reports Find Fault With High-Stakes Testing,” Jan. 8, 2003).

For his study, co-written with Audrey L. Amrein, Mr. Berliner compared states’ academic gains against the national average. He and Ms. Amrein found that most of the high-pressure states saw decreases in 4th grade math scores after adopting their testing programs. At the 8th grade level, a majority of high-stakes states gained relative to the national average.

Referring to Mr. Braun, Mr. Berliner added: “He was able to find results, but my guess is that our study and everybody else’s is still going to be subject to criticism.”

A version of this article appeared in the January 28, 2004 edition of Education Week as Study Offers Mixed Results On Impact of High-Stakes Tests

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and K-12 education jubs at the EdWeek Top School Jobs virtual career fair.
Mathematics K-12 Essentials Forum Helping Students Succeed in Math
Student Well-Being Live Online Discussion A Seat at the Table: The Power of Emotion Regulation to Drive K-12 Academic Performance and Wellbeing
Wish you could handle emotions better? Learn practical strategies with researcher Marc Brackett and host Peter DeWitt.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion The Stunning Resignation of UVA President Jim Ryan—and Why It Matters
The university president’s departure is more than just a headline. It’s a lesson in leadership.
2 min read
Opinion Licensed Not for Reuse Wait What FCG
Canva
School & District Management In Their Own Words This Custodian Got Students to Stop Vandalizing and Take Pride in Their School
Andy Markus, the 2025 Education Support Professional of the Year, helped boost behavior and engagement in his Utah district.
5 min read
Andy Markus, the head custodian at Draper Park Middle School, in Draper, Utah, sits for a portrait during the National Education Association's 2025 Representative Assembly in Portland, Ore., on July 3, 2025. Markus was named the 2025 NEA Education Support Professional (ESP) of the Year.
Andy Markus, the head custodian at Draper Park Middle School, in Draper, Utah, sits for a portrait during the National Education Association's 2025 representative assembly in Portland, Ore., on July 3, 2025. Markus was named the 2025 NEA Education Support Professional of the Year for his mentorship of students.
Kaylee Domzalski/Education Week
School & District Management What the Research Says About School Boards: How Much Conflict Really Is There?
Plus, how competitive are board elections? How much do teachers' union endorsements matter?
7 min read
Houston ISD's appointed school board votes on the "District of Innovation" status during their monthly work session meeting at HISD Central Office on Sept. 7, 2023 in Houston.
Houston's appointed school board takes a vote during a meeting on Sept. 7, 2023 in the district's central office. A number of studies from recent years have answered questions about school boards' makeup, how competitive board elections are, whether conflict is on the rise, and more.
Karen Warren/Houston Chronicle via AP
School & District Management Opinion How a Weekly Email to My Staff Made Me a Better District Leader
Writing helps make sense out of what feels messy and focus us on what's most important.
George Philhower
5 min read
Blue hand holding red pen.
DigitalVision Vectors/Getty + Education Week