Opinion
Education Letter to the Editor

Achievement Scores Often Misinterpreted

October 02, 2012 2 min read

To the Editor:

The article “Principals Criticized on Teacher-Retention Decisions” (Aug. 8, 2012) contains the following statement about results issued in a report by TNTP, formerly The New Teacher Project: “Of the teachers studied, the group identified a subset of about 20 percent as ‘irreplaceables’ because their students made two or three more months’ worth of academic progress compared [with] those taught with the average teacher in the district.” The statement is fundamentally flawed and represents a common misunderstanding fostered by the testing industry, which prefers to report results in months and years of achievement when all they have is the number of items scored as correct on a particular test: the raw score.

We do not know what an achievement test measures. A person’s score on a test of general mental ability and the person’s score on an achievement test, which is supposed to reflect what a student has learned in school, are almost interchangeable. A person scoring high on one test will also score high on the other test, even though the content of the two tests is quite different. A Texas researcher recently reported that the pattern of results on 100,000 achievement tests was best explained by a “latent trait” he called “test-taking ability"; test results had little to do with instruction. Depending on the scale and the test, a “month” of instruction may mean little more than one or two additional correct items.

In the literature, I have found little or no empirical support for taking the raw score for a given grade and transforming raw scores into months and years. There is no evidence to show that a change in raw scores correlates with months of instruction, much less effectiveness of instruction.

Items on achievement tests are supposed to represent aspects of the state’s educational standards. If we look at raw scores, we cannot tell what it means educationally that a student got two or three more items correct. Which of the standards do those additional items represent? What can the student taught by an “irreplaceable” teacher do that one taught by an average teacher cannot do? Moreover, each specific state standard is represented on the typical test by too few items to be a statistically meaningful sample of mastery of that domain of knowledge. If the pattern of correct items is highly similar in students taught by an “irreplaceable” teacher, how do we distinguish that result from coaching?

The uncertainties of achievement testing are so great we must question their uses, especially when the language in which results are reported is highly misleading.

Murray Levine

Distinguished Service Professor, Emeritus

Department of Psychology

State University of New York at Buffalo

Buffalo, N.Y.

A version of this article appeared in the October 03, 2012 edition of Education Week as Achievement Scores Often Misinterpreted

Let us know what you think!

We’re looking for feedback on our new site to make sure we continue to provide you the best experience.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Future of Work Webinar
Digital Literacy Strategies to Promote Equity
Our new world has only increased our students’ dependence on technology. This makes digital literacy no longer a “nice to have” but a “need to have.” How do we ensure that every student can navigate
Content provided by Learning.com
Mathematics Online Summit Teaching Math in a Pandemic
Attend this online summit to ask questions about how COVID-19 has affected achievement, instruction, assessment, and engagement in math.
School & District Management Webinar Examining the Evidence: Catching Kids Up at a Distance
As districts, schools, and families navigate a new normal following the abrupt end of in-person schooling this spring, students’ learning opportunities vary enormously across the nation. Access to devices and broadband internet and a secure

EdWeek Top School Jobs

Speech Therapists
Lancaster, PA, US
Lancaster Lebanon IU 13
Elementary Teacher
Madison, Wisconsin
One City Schools
Elementary Teacher - Scholars Academy
Madison, Wisconsin
One City Schools

Read Next

Education Obituary In Memory of Michele Molnar, EdWeek Market Brief Writer and Editor
EdWeek Market Brief Associate Editor Michele Molnar, who was instrumental in launching the publication, succumbed to cancer.
5 min read
Education Briefly Stated Briefly Stated: December 9, 2020
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of articles from the previous week that you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of stories from the previous week that you may have missed.
8 min read