Sloppy, sloppy, sloppy. In reasoning and logic, perhaps the only thing worse than a lack of critical-thinking skills--whatever those are--is a prolific use of what I call “sloppy thinking skills.” And nowhere are examples of sloppy reasoning more abundant than in current calls to restructure education by abolishing standardized testing.
Questioning the logic of the testing abolitionists is a risky endeavor. Criticism of that cause implies approval of the intellectual slavery to tests that those in the movement oppose. In extreme cases, proponents of testing are painted as promoters of racial inequality, perpetuators of economic injustice, and purveyors of fraudulent goods and services.
But perils aside, careful examination of the abolitionists’ claims and aims is in order. Such scrutiny, we must hope, will stimulate further discussion, dissension, and development in the technology of testing.
Criticism of testing has lately been widespread and multifaceted. In recent months, for example, readers of Education Week have learned of a movement to abolish assessment in early grades and of a coalition formed to combat standardized testing. They have also been offered arguments for reconsidering standards and assessments altogether.
There may be solid grounds for such reappraisal. But the rationale advanced in recent discussions is ill-conceived and poorly defended--the product of sloppy logic. The American public has yet to be presented a substantial, well-conceived argument for abandoning testing.
Sloppy reasoning has at least three characteristics: It employs grandiose but vacuous rhetoric; it makes careless inferences regarding cause and effect; and it mistakes faith for fact. While the mere presence of this unholy trinity in the current debate is worrisome, more troubling is the possibility that it could actually influence movement toward reform.
The abolitionists’ use of rhetoric is most noticeable in their attempts to capture the moral high ground through emotional appeal. For example, on first glance, what right-thinking person would not want “schools with high standards,” an “attitude of excellence,” “genuine accountability,” more “quality than quantity,” “authentic evaluation,” or “empowerment for teachers”?
But on second glance, other questions loom: “What in the world is ‘genuine accountability’?” and “How does ‘genuine accountability’ differ from regular accountability?” A call for “authentic evaluation” implies that “counterfeit evaluation” must also exist. Who are the evaluation counterfeiters? Who is charged with distinguishing the counterfeit from the authentic? Should there be criminal penalties for counterfeit evaluation?
What does all of this jargon really mean? Are any of these chic phrases meant for any purpose except to make us feel bad about the state of evaluation and to make us feel good about the poorly defined alternatives? I don’t think so. In fact, all of the moaning and gloom-saying about current practice suggests a new moniker around which the abolitionists can rally: “genu-whine assessment.”
In fairness to the movement, however, we must ask if the whining is justified--if testing is indeed what ails American education. Unfortunately, the argumentation offered in support of this position is mired in careless inferences about cause and effect.
Generally, the abolitionists rightly note that testing is not the cause of higher achievement. No one seriously contends that it is. But neither is it logical to argue that testing is the cause of lower achievement. Assessment is simply an attempt to gauge the current state of affairs. The too-frequently invoked medical analogy remains useful in this context, especially given a widespread perception that the education system is ailing: We take the patient’s temperature more frequently when there appears to be some reason to believe that something is wrong. Such monitoring, we think, will provide a hint about why the patient is sick and tell us whether appropriate therapy is leading to better health.
The cause-and-effect mistake runs even deeper. The researcher Grant Wiggins, for example, has commented that “in the last 20 years, the most massive investment in testing ever undertaken has coincided with a palpable decrease in the quality of education.” Sure. And also in the last 20 years, average cholesterol levels have risen, Communist governments have begun to fall, and Nelson Mandela has been freed. Ah, the power of testing!
The most plausible cause-and-effect inference regarding testing and educational quality has been overlooked by the abolitionists: that the patient’s temperature is being taken more frequently because other available data suggest that there is a problem. To hypothesize that testing could undermine reform efforts--as did the coalition of groups that in January released a statement opposing standardized assessment and backing “genuine accountability"--is to wholly misunderstand the problem.
Interestingly, the same abolitionists who claim that a love of testing is the root of all educational evils propose as an alternative merely to substitute different kinds of testing. The newest is “performance assessment,” which, we are told, is more “authentic” or “genuine.” Or maybe it’s “higher order,” or “holistic.” In any event, the opponents of standardized assessment are urging the testing companies and the public to read what Ruth Mitchell calls “the handwriting on the wall,” and support the use of something else, something “more real.” Indeed. If anything, the public has already read the handwriting on the wall and is reacting negatively because it contains misspellings and poor grammar.
Which brings us to the third characteristic of sloppy reasoning: mistaking faith for fact. If this sort of thinking isn’t acceptable in teaching about matters as significant as the origins of the world, it shouldn’t be permitted in areas of arguably lesser importance, either.
The abolitionists would ask us to be educationally healed by faith in their dogma. While promoting higher standards for education, they suggest that “students with high standards are diligent, thoughtful, engaged, persistent, and thorough--no matter what they learn.” That pronouncement features a long string of adjectives expressing a nice sentiment. Such emotive language might compel the reader to blindly accept the statement’s underlying assumption--that it doesn’t matter what students learn. Are we to accept this premise on faith? Is it sound?
A. Graham Down, executive director of the Council for Basic Education and a signer of the “statement on genuine accountability,” asserts that alternatives are needed because the traditional tests only measure “knowledge.”
Now, I suppose I could be accused of being old-fashioned here, but somehow I’m fixated on the notion that knowledge is still fundamental to learning. Education just wouldn’t be the same without it.
What about allegations that testing has been particularly harmful for children in the early grades and that, in the words of one abolitionist, “in some schools they spend more time in testing than they do in actual classrooms”? These are serious charges--if true. Such claims must be supported if we are to believe them; we can’t simply take them on faith.
Perhaps all of these lapses in reasoning can be explained by the spread of what might be called “evalophobia.” In proposing that testing be banned, the abolitionists are also implicitly suggesting that we renounce any means of exploring the link between quality of instruction and quality of performance. Of course, standardized tests don’t measure everything that is valuable or interesting about student achievement. But they do measure some of what is valuable and interesting in an extremely efficient and accurate way.
Reasonable inferences from high scores on a standardized test include the following: The high-scoring students seem to be grasping what is taught; they are performing better on the tested material than some other students; and the instruction has been appropriate. Reliance on the proposed “genuine,” “empowering” assessments would render such conclusions as the first two tenuous and the third impossible. In fact, much of the rhetoric surrounding alternative assessment leaves individual students entirely culpable for their educational shortcomings. We should not, for fear of the information provided by testing, deprive ourselves and our students of one of the few sources of objective data for evaluating our own performance.
We should all recognize that we have not achieved technical perfection in assessment; we must also admit that existing instruments address only a portion of what is useful for both our students and ourselves.
But we should not jettison the tests we have because they don’t do everything we wish they could. At minimum, tests do provide important information about some aspects of learning, as everyone should acknowledge. And we should develop new devices to assess other facets of students’ intellectual growth.
