F. Allan Hanson is a professor of anthropology at the University of Kansas. This essay is derived from his recent book, Testing Testing: Social Consequences of the Examined Life.
1. Would you be willing to pay a little more to help your children and grandchildren have a brighter future?
2. Do you believe it is your patriotic duty to entrust more of your money to a pack of moochers, double-talkers, and deadbeats?
Tests are an outstanding example of the same phenomenon. Not only does the phrasing of particular test questions draw forth certain answers but, more generally, different types of testing have important consequences. As every teacher knows, multiple-choice and essay tests generate significantly different study techniques and learning experiences. Jacques Barzun has even suggested that the preoccupation with doing well on standardized tests has literally molded the way young people in America think. They have better developed cognitive abilities to recognize random facts than to construct patterns or think systematically, he argues, because the former skill is favored and rewarded by the multiple-choice format of the standardized tests that they encounter so frequently.
In Stories in an Almost Classical Mode, Harold Brodkey writes of his childhood: " ... but I did well in school and seemed to be peculiarly able to learn what the teacher said--I never mastered a subject, though--and there was the idiotic testimony of those peculiar witnesses, I.Q. tests: Those scores invented me. Those scores were a decisive piece of destiny in that they affected the way people treated you and regarded you; they determined your authority; and if you spoke oddly, they argued in favor of your sanity.’'
In a very real sense, tests have invented all of us. They play an important role in determining what opportunities are offered to or withheld from us, they mold the expectations and evaluations that others form of us (and we form of them), and they heavily influence our assessments of our own abilities and worth. Therefore, although testing is usually considered to be a means of appraising qualities that are already present in a person, in actuality the individual in contemporary society is not so much measured by tests as constructed by them.
Similarly, tests produce many of the qualities and abilities that they supposedly measure. Nowhere is this clearer than with the concept of “intelligence.’' Although psychologists debate precisely what intelligence is, in the popular mind it has to do with general mental ability; perhaps it is best stated as the ability to learn. This is often fleshed out by associating intelligence with three attributes: 1) It is a single thing; 2) it comes in varying quantities, such that different people have different amounts of it; and 3) the amount of intelligence possessed by each individual is fixed for life.
The conventional view of intelligence derives largely from practices of intelligence testing. First, the idea that intelligence is a single thing is rooted in the fact that the results of intelligence tests are often expressed on a single scale, such as I.Q., even when the test itself consists of several distinct parts. Where there is a single score, it is easy to conclude that some single thing must exist to which that score refers. The second attribute--that intelligence is quantitative, and that some people have more of it than others--derives from the practice of reporting intelligence-test scores on numerical scales. Only quantitative phenomena may be expressed in numbers. And when those numbers vary from one person to another, so must the amount of intelligence that the numbers represent. Finally, the notion that the amount of intelligence possessed by each individual is fixed for life stems from the belief that intelligence tests measure not so much what one already knows as one’s ability to learn--something that is thought to be “hard-wired’’ in a person. This view is reinforced by those companies that include, in the materials they consider for hiring and promotion, scores from S.A.T. or A.C.T. tests taken years before.
The process whereby intelligence tests create the concept of intelligence may be readily observed if we engage in an admittedly tongue-in-cheek thought experiment of constructing a new test and imagining its consequences. Let us call our test, simply, the New Intelligence Test, or NIT. It is intended to respond to criticisms of current tests by paying more attention to the practical aspects of intelligence used in everyday life, and to sample more widely from the scope of intelligence than just quantitative and verbal abilities. Hence the NIT consists of nine sections:
1) A name-recall scale tests ability to remember the names of persons to which the subject has just been introduced.
2) A mathematics section tests the subject’s ability to do problems of arithmetic and algebra.
3) The first-impression scale invites a panel of ordinary people to evaluate the personableness of subjects by simply looking at them.
4) In the exposition-of-ideas section, the subject is given five minutes to read a complex idea--such as Rousseau’s distinction between self-love (amour de soi) and selfishness (amour-propre)--and 30 minutes to present a clear and accurate written account of it, with original examples.
5) The small-talk scale evaluates subjects’ ability to carry on an interesting conversation with someone they have just met.
6) A B.S.-ing scale assesses skill at participating in a discussion with two other people on a topic about which the subject knows nothing.
7) In the follow-the-directions scale, the subject is told once, at the speed of ordinary conversation, to do a task that consists of six distinct steps, and is evaluated on how well the task is accomplished.
8) The adult sports scale evaluates the subject’s ability to play golf or tennis, with suitable adjustments for male and female subjects.
9) Finally, the socioeconomic, or S.E.S. scale is a simple rating of subjects according to family income.
A composite score is generated from the results of the NIT’s nine sections.
What ability or human capacity is tested by the NIT? An operational definition of intelligence used by numerous psychologists is that intelligence is that mental capacity which is measured by intelligence tests. On that precedent, one would say that the NIT tests the skills or wits used in taking the NIT, no more and no less. This is certainly nothing inconsequential, for were the appropriate studies to be done, it would doubtless turn out that high NIT scores correlate positively (probably more positively than I.Q. scores) with desirable social outcomes such as success in the university, in business or professional life, and election to public office. But it is also obvious that what the NIT tests is not a single quality or capacity of persons. It is rather a set of distinct qualities, which have been measured by the several sections of the NIT and combined into a single score for convenience in reporting NIT results.
But assume now that the NIT were to catch on in a big way--that it came, for example, to be widely used for college and graduate-school admissions and for hiring and promotion purposes by law firms, government, and corporations. In such an event, the different abilities measured by the NIT would not remain static. People would spare no effort in preparing for the test, in the hope of achieving the rewards awaiting those who excel on it. They would bone up on arithmetic and algebra, they would master techniques for remembering the names of strangers, they would practice B.S.-ing, they would take golf and tennis lessons, they would groom themselves to appear more likable on first sight.
High school and college curricula would shift in the direction of more training in the areas covered by the NIT. (If they did not, irate parents would demand to know why their children were not being taught something useful.) Stanley Kaplan and Princeton Review would explode into the marketplace with courses that promise dramatic improvement in one’s NIT scores. One side effect would even be to diminish the national debt as people report inflated income in order to improve their children’s showing on the NIT/S.E.S. scale--and then have to pay taxes on it.
All of this dedicated effort would have a palpable effect. Although the NIT obviously measures several quite different abilities, people would knit them together as they strive to improve them all in order to raise their NIT scores. They would begin to imagine these several abilities to be one. They would name it ... perhaps “NITwit.’' Given its importance for success in life, it would be valued as a thing of great significance. People would worry about how much of it they possess; they would envy evidence of its abundance in their contemporaries and look for promising signs of it in their children.
Not only would a new mental category swim into the social consciousness. The empirical amount of it possessed by individuals would literally increase as, in preparing for the NIT, they hone their skills at following directions, playing golf, expounding on ideas, small talk, and the rest of it. And, of course, as individuals increase these skills, average NIT scores would go up. There would be rejoicing in the land as today’s NIT scores exceed those achieved in the past or by test-takers in other countries ... until, perhaps, an apogee is passed and national consternation about declining NIT scores sets in.
Given all these transformations and developments, it is fair to say that NITwit would become a new, singular, personal trait--an objective reality literally constructed by NIT testing. Perhaps the ultimate development (and the ultimate absurdity, but it unquestionably would happen) would be when rival tests are marketed that claim to measure NITwit faster, cheaper, or more accurately than the NIT.
What happened in our thought experiment has been the experience of “intelligence’’ in the real world. Because of intelligence tests, several different abilities (to do mathematical problems, to comprehend tests, to compare shapes, to sort ideas or objects into classes, to define words, to remember historical events, and to do all of these things rapidly) have been lumped together to create a unitary mental characteristic called “intelligence.’' It is a quality of great importance because it often determines who will receive educational and career opportunities and other coveted social rewards. Nevertheless, its formation in the crucible of intelligence tests renders it no less an invention than the intelligence associated with machines. Both are “artificial.’'
