Opinion
Education Letter to the Editor

Achievement Scores Often Misinterpreted

October 02, 2012 2 min read
  • Save to favorites
  • Print

To the Editor:

The article “Principals Criticized on Teacher-Retention Decisions” (Aug. 8, 2012) contains the following statement about results issued in a report by TNTP, formerly The New Teacher Project: “Of the teachers studied, the group identified a subset of about 20 percent as ‘irreplaceables’ because their students made two or three more months’ worth of academic progress compared [with] those taught with the average teacher in the district.” The statement is fundamentally flawed and represents a common misunderstanding fostered by the testing industry, which prefers to report results in months and years of achievement when all they have is the number of items scored as correct on a particular test: the raw score.

We do not know what an achievement test measures. A person’s score on a test of general mental ability and the person’s score on an achievement test, which is supposed to reflect what a student has learned in school, are almost interchangeable. A person scoring high on one test will also score high on the other test, even though the content of the two tests is quite different. A Texas researcher recently reported that the pattern of results on 100,000 achievement tests was best explained by a “latent trait” he called “test-taking ability"; test results had little to do with instruction. Depending on the scale and the test, a “month” of instruction may mean little more than one or two additional correct items.

In the literature, I have found little or no empirical support for taking the raw score for a given grade and transforming raw scores into months and years. There is no evidence to show that a change in raw scores correlates with months of instruction, much less effectiveness of instruction.

Items on achievement tests are supposed to represent aspects of the state’s educational standards. If we look at raw scores, we cannot tell what it means educationally that a student got two or three more items correct. Which of the standards do those additional items represent? What can the student taught by an “irreplaceable” teacher do that one taught by an average teacher cannot do? Moreover, each specific state standard is represented on the typical test by too few items to be a statistically meaningful sample of mastery of that domain of knowledge. If the pattern of correct items is highly similar in students taught by an “irreplaceable” teacher, how do we distinguish that result from coaching?

The uncertainties of achievement testing are so great we must question their uses, especially when the language in which results are reported is highly misleading.

Murray Levine

Distinguished Service Professor, Emeritus

Department of Psychology

State University of New York at Buffalo

Buffalo, N.Y.

A version of this article appeared in the October 03, 2012 edition of Education Week as Achievement Scores Often Misinterpreted

Events

Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Building Teacher Capacity for Social-Emotional Learning
Set goals that support adult well-being and social-emotional learning: register today!


Content provided by Panorama
Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Tiny Wrists in Cuffs: How Police Use Force Against Children
An investigation finds children as young as 6 and a disproportionate amount of Black children have been handled forcibly by police officers.
15 min read
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Nam Y. Huh/AP
Education Gunman in 2018 Parkland School Massacre Pleads Guilty
A jury will decide whether Nikolas Cruz will be executed for one of the nation’s deadliest school shootings.
3 min read
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Amy Beth Bennett/South Florida Sun Sentinel via AP
Education Briefly Stated: October 20, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Gunman in Parkland School Massacre to Plead Guilty
The gunman who killed 14 students and three staff members at a Florida high school will plead guilty to their murders, his attorneys said.
4 min read
Parkland school shooter Nikolas Cruz is sworn in before pleading guilty, Friday, Oct. 15, 2021, at the Broward County Courthouse in Fort Lauderdale, Fla., on all four criminal counts stemming from his attack on a Broward County jail guard in November 2018, Cruz's lawyers said Friday that he plans to plead guilty to the 2018 massacre at a Parkland high school.
Parkland school shooter Nikolas Cruz is sworn in before pleading guilty, Friday, Oct. 15, 2021, at the Broward County Courthouse in Fort Lauderdale, Fla., on all four criminal counts stemming from his attack on a Broward County jail guard in November 2018, Cruz's lawyers said Friday that he plans to plead guilty to the 2018 massacre at a Parkland high school.
Amy Beth Bennett/South Florida Sun Sentinel via AP