School & District Management

Study Offers Mixed Results On Impact of High-Stakes Tests

By Debra Viadero — January 28, 2004 3 min read
  • Save to favorites
  • Print

Efforts in more than half the states to tie major consequences to students’ test scores are translating into academic gains, according to the latest in a series of studies on the policy approach known as high-stakes testing.

The report, “Reconsidering the Impact of High-Stakes Testing,” is available from the Education Policy Analysis Archives.

Or then again, maybe they’re not.

The study, published this month in the online journal Education Policy Analysis Archives, draws on eight years of national testing data to compare states with traditional “low stakes” testing policies against those with “high stakes” systems. Under high-stakes policies, students’ scores are used to decide which teachers or schools win cash bonuses, whether students graduate or move on to the next grade, or what schools are subject to takeover by their districts or states.

The report follows half a dozen other studies over the past year that have used similar techniques to evaluate the effects of such accountability systems. (“Study Finds Higher Gains in States With High-Stakes Tests,” April 16, 2003).

“I was intrigued by the fact that different researchers with different ideological stances were coming to different conclusions from the same data,” said Henry I. Braun, the author of the new study. “I was also motivated by a sense that the world of research is very complex and we are not, in our research worlds, respectful enough of that complexity.”

Sorting It Out

Mr. Braun, a statistician with the Princeton, N.J.-based Educational Testing Service, used four different methods to compare changes in states’ scores on National Assessment of Educational Progress mathematics tests between 1992 and 2000.

The comparisons pitted the 18 states that some previous researchers have identified as having high-stakes systems against 32 with lower-pressure accountability systems.

Looking first at overall changes in the states’ 4th and 8th grade test scores over that period, Mr. Braun, like most of his predecessors, found that students’ academic gains were greater in states, such as Texas and North Carolina, that had high- pressure testing systems.

What’s more, he said, the trend could not be explained by statistical errors or the fact that some of the states showing the biggest improvements had also been excluding growing percentages of special education students from the tests.

In the 4th grade, the difference in mean scores between the high-stakes and low-stakes states was 4.3 score points; in 8th grade, it was 3.99 score points.

The opposite occurred, though, when Mr. Braun took a look at how cohorts of students fared on the tests over time. (He compared 4th graders’ scores with the 8th grade scores in the same states four years later.)

That time around, the improvements in academic achievement were greater—albeit to a lesser degree—in the states with low-pressure testing systems.

Mr. Braun said the differing results didn’t surprise him.

“You cannot look at high-stakes testing in isolation from other things going on in the state,” he said. “Many education reforms can be assisted or thwarted by other education reforms going on at the same time.”

In an effort to take a broader look, Mr. Braun reconfigured the data to factor in a measure that rated states on their education activism. It assigned states grades based on whether they had enacted—or were about to enact—22 school improvement efforts, such as professional standards for teachers or subject-matter standards.

But he found little correlation between the level of states’ education activism and their students’ test-score changes over time.

Mr. Braun also looked at changes in scores for the bottom 25 percent of students in each of the states. Gains were greater in the states that put more pressure on students or schools for test-score improvements.

“All of this is about which states you include and which states you do not include,” said David C. Berliner, an education professor at Arizona State University in Tempe whose own study on high-stakes testing helped spark the spate of research on the subject. (“Reports Find Fault With High-Stakes Testing,” Jan. 8, 2003).

For his study, co-written with Audrey L. Amrein, Mr. Berliner compared states’ academic gains against the national average. He and Ms. Amrein found that most of the high-pressure states saw decreases in 4th grade math scores after adopting their testing programs. At the 8th grade level, a majority of high-stakes states gained relative to the national average.

Referring to Mr. Braun, Mr. Berliner added: “He was able to find results, but my guess is that our study and everybody else’s is still going to be subject to criticism.”

A version of this article appeared in the January 28, 2004 edition of Education Week as Study Offers Mixed Results On Impact of High-Stakes Tests

Events

Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion ‘This Is Not What We Signed Up For’: A Principal’s Plea for More Support
School leaders are playing the role of health-care experts, social workers, mask enforcers, and more. It’s taking a serious toll.
Kristen St. Germain
3 min read
Illustration of a professional woman walking a tightrope.
Laura Baker/Education Week and uzenzen/iStock/Getty
School & District Management Letter to the Editor Educators Must Look to History When They Advocate for Changes
Educators and policymakers must be aware of the history of ideas when making changes in education, says this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.
iStock/Getty
School & District Management Letter to the Editor Reconsidering Causes of Principal Burnout
The state and federal governments are asking us to implement policies that often go against our beliefs, says this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.
iStock/Getty
School & District Management Teachers Want Their Administrators to Teach. Here's Why
Principals and other education administrators should even be required to spend time teaching in the classroom, according to teachers responding to an EdWeek query.
Hayley Hardison
4 min read
Teacher Principal 11122021 1310106400
E+/Getty