In an investigative report that was supported by charts and graphs, The New York Times on Oct. 11 revealed how New York State officials ignored warnings from experts for more than a decade about the way tests were being misused to create the illusion of progress (“On New York School Tests, Warning Signs Ignored”). The front-page story was a case study of assessment legerdemain in action.
What it showed was that a singular obsession with test scores had distorted the entire assessment process, undermining confidence in the claims made by Mayor Michael Bloomberg and Chancellor Joel Klein. In publishing the story, the Times performed a public service. Kudos to Jennifer Medina, the chief reporter, who was assisted by Robert Gebeloff and Elissa Gootman, and to Jack Begg, who contributed research.
To understand the article, it’s necessary to realize that tests by themselves do not possess validity. It’s the inferences made about what teachers have taught and what students have learned that are valid or invalid. The trouble is that few people in power took the time to ask the proper questions about the evidence used to support these inferences. They preferred instead to view scores as sacrosanct. The scores determined which schools closed, which students were promoted, and which teachers and administrators received bonuses. In short, they took on a life of their own.
Although testing is an indispensable part of the instructional process, it has to be carried out in a way that measures real learning. Otherwise, it becomes a game that is subject to abuse. It’s important to bear in mind that no perfect test has ever been designed, and none ever will. There will always be fair criticism. The goal is to come up with a test that is as close as possible to the ideal. Instead, New York chose to buy solely into numbers-based accountability.
The price it is now paying is deep skepticism from the public at a time when their support is desperately needed. Yet this outcome was totally predictable. The more any quantitative indicator is used for decision making, the more it will be subject to corruption, and the more it will corrupt the very process it is intended to monitor. Unless Campbell’s Law is repealed, what New York is undergoing will be repeated in other states again and again.