Education

Test-Score Averages Don’t Tell Whole Story About Schools, Study Says

By Debra Viadero — April 21, 2004 3 min read
  • Save to favorites
  • Print

A new study suggests that the methods most states are using to judge schools’ academic progress may produce an incomplete—and possibly misleading—picture of the jobs the schools are doing.

Read “Individual Growth and School Success,” from the Northwest Evaluation Association. (Requires free registration.)

Based on two years of test-score data for 230,000 students in 22 states, the study was released last week by the Northwest Evaluation Association, a nonprofit group that provides testing help to 1,200 member school districts.

The findings are noteworthy, researchers said, because they come as states are gearing up to comply with the federal testing requirements outlined in the No Child Left Behind Act. Under the law, states are required to use test scores to determine whether schools are making “adequate yearly progress” and to hold schools accountable when students’ scores fail to rise.

To meet that requirement, most states are using simple test-score averages, said G. Gage Kingsbury, the director of research for the Portland, Ore.-based group. The problem with that method, he said, is that it might not reveal much about how much students are learning—in other words, their academic growth over time.

For example, two schools might have identical test-score averages and thus earn similar ratings under the federal law. What the numbers might not reveal is that students at one school started out farther behind students at the other school.

“There’s not too much difference between these schools in terms of NCLB,” said Mr. Kingsbury. “But if I was a parent, I’d want my child in the school that’s causing more growth in students’ learning.”

Similar, But Different

To explore how such a growth-oriented evaluation approach might play out, the Oregon researchers analyzed scores for students in grades 3-8 from 723 of the association’s member districts. Students took the tests in the spring of 2002 and 2003.

The goal was to gauge the amount of unexpected score gains that students made from one year to the next.

As they had predicted, the researchers found that schools with similar performance ratings actually varied widely in the test-score gains their students were making.

More than 20 percent of the schools that were rated high-performing by their states fell into the bottom quarter when measured by the researchers’ test-score-growth index.

The reverse was also true: Several schools with low average test scores had score gains as large as those of the highest-performing schools. The differences between similarly ranked schools sometimes added up to as much as two-thirds of a year in a students’ academic growth.

“If a low-status school started out with students with low status and yet moved them dramatically forward, I would argue that might actually be a success story,” Mr. Kingsbury said.

Under provisions of the federal law that allow students to transfer out of failing schools, though, that school could lose students, Mr. Kingsbury said. The students who transfer could also conceivably end up at higher-ranked schools where they might make less academic progress than they did at the schools they left behind.

Likewise, Mr. Kingsbury argued, high- performing schools that seemed to be doing little more than nudging students’ learning along have no incentive to work harder under the evaluation methods that states are using now.

Students at high-performing schools didn’t “top out” on the tests because the tests were given on computers that adapted the test questions to students’ actual test performance, he added. In other words, the programs assigned progressively harder questions to students as they answered more questions correctly on particular topics.

The drawback to such growth or “value added” measurement techniques, Mr. Kingsbury said, is that tracking students’ progress is difficult when students move from school to school.

To address that problem, he said, states should assign students identification numbers that can be tracked electronically through their testing systems.

Still, he said, combining the growth measure with a measure of average test scores might give a clearer picture of the kinds of jobs that schools are doing in enhancing students’ academic progress than the methods that states rely on now.

Currently, Mr. Kingsbury said, most states turn to academic-growth measures only after a school has been labeled failing.

Events

Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Data Webinar
Using Integrated Analytics To Uncover Student Needs
Overwhelmed by data? Learn how an integrated approach to data analytics can help.

Content provided by Instructure
School & District Management Live Online Discussion Principal Overload: How to Manage Anxiety, Stress, and Tough Decisions
According to recent surveys, more than 40 percent of principals are considering leaving their jobs. With the pandemic, running a school building has become even more complicated, and principals' workloads continue to grow. f we

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Gunman in 2018 Parkland School Massacre Pleads Guilty
A jury will decide whether Nikolas Cruz will be executed for one of the nation’s deadliest school shootings.
3 min read
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Amy Beth Bennett/South Florida Sun Sentinel via AP
Education Briefly Stated: October 20, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Gunman in Parkland School Massacre to Plead Guilty
The gunman who killed 14 students and three staff members at a Florida high school will plead guilty to their murders, his attorneys said.
4 min read
Parkland school shooter Nikolas Cruz is sworn in before pleading guilty, Friday, Oct. 15, 2021, at the Broward County Courthouse in Fort Lauderdale, Fla., on all four criminal counts stemming from his attack on a Broward County jail guard in November 2018, Cruz's lawyers said Friday that he plans to plead guilty to the 2018 massacre at a Parkland high school.
Parkland school shooter Nikolas Cruz is sworn in before pleading guilty, Friday, Oct. 15, 2021, at the Broward County Courthouse in Fort Lauderdale, Fla., on all four criminal counts stemming from his attack on a Broward County jail guard in November 2018, Cruz's lawyers said Friday that he plans to plead guilty to the 2018 massacre at a Parkland high school.
Amy Beth Bennett/South Florida Sun Sentinel via AP
Education California Makes Ethnic Studies a High School Requirement
California is among the first in the nation to require students to take a course in ethnic studies to get a diploma starting in 2029-30.
4 min read
FILE - In this Jan. 22, 2020, file photo, Democratic Assembly members, from left, James Ramos, Chris Holden Jose Medina, and Rudy Salas, Jr., right, huddle during an Assembly session in Sacramento, Calif. Medina's bill to make ethnic studies a high school requirement was signed into law by California Gov. Gavin Newsom on Friday, Oct. 8, 2021. (AP Photo/Rich Pedroncelli, File)