Dear Diane,
The notion of good medical care as “test prepping” is delightfully bizarre, but maybe no less so than traditional forms of test-prepping? Perhaps Richard Rothstein is correct that it would have a greater impact on test score gaps.
Traditional psychometrics is filled with problems built into the excellent history you’ve offered us, Diane. It presumes that scores fit a curve, and that there is a norm, usually labeled 100 or expressed in percentiles with 50 percent below, 50 percent above grade or age “level”, with a particular cluster toward the center. All very precise.
How come half the kids are still below grade level after years of school improvement, I was once asked? Half were below because that’s how we defined “grade level”, I explained. If more kids started doing better—via test prep or actual skill—they’d renorm the tests. Scores were just a “promise” about where one stood in the rank ordering of all students who might take this test at this particular point in their school career.
Recent attempts to change the scoring method so that it reflects benchmarks of skill rather than percentiles is a positive step, even if you and I disagree regarding current definitions of basic, proficient, etc. The big plus (or weakness) is that they now rest on human judgment, not mathematical formulas.
Incidentally, in response to some of the critics who worried about my two math examples: they weren’t intended to “prove” anything except that we know precious little about why “right” answers appeal to some people more often than to other people. I included that duo precisely because they puzzle me, and why I appreciated readers’ efforts to explain them, and Jay Rosner’s replies.
What I’ve discovered is that both the kids who get the answers right and the ones who get them wrong can often give persuasive reasons for their selection. Sometimes—and over the years I’ve collected a host of these—the right answer is, upon closer examination, even clearly wrong. But what they are never wrong about is who gets which ones right! Are you following this?
When asked on a 3rd grade reading test, for example, how the children felt when the trucks came and cut down the trees to widen the highway by their house, only the middle-class kids said “sad"—which was the “right” answer. Many of the others said “excited”. I know that’s a particularly stupid question, but the division was startling.
Ditto for a familiar IQ question on what to do if you lose a friend’s ball. Middle-class kids are more likely to say “buy him a new one” and working-class kids to say “tell the teacher”. In fact, the latter are the least likely to tell the teacher, but more likely to believe that was the right answer.
Also interesting is how hard it is to guess whether items come from IQ or reading or math tests. I’ve tried random items out on audiences and gotten random answers.
Rosner’s point, among others, was that if ETS (Educational Testing Service) has historically played around with items to ensure that male scores more properly matched or surpassed female scores, it could do so for race. Yet, he discovered, that the pre-tested items on which blacks more often got answers right were virtually never used—although they existed. The two items described were among that pool.
Democracy requires us to “act as if” the views of every single citizen are of equal importance. Not equally “correct” or “wise"—just equally important. And that each has a “logic” that the community would do well to acknowledge and make sense of.
That’s what I rest my case on for democracy writ large, and writ small. I take it for granted, or tried to, that the stupidest questions or answers were important to understand. Doing so did not always lead to new truth. But it amazes me how often it broadened my understanding and that of others in the class. After a while it became possible to “outlaw” the phrase “it’s obvious” in class! Jean Piaget’s work helped us see precisely how difficult it was to convince someone else of “the obvious” when they were viewing it from a different perspective of age, experience, temperament.
A youngster, studying a world map in the hall, asked me (shyly) why the West Indies was in the east and the East Indies in the west? It startled me into noticing an “obvious” misnomer stemming from the perspective of the maps we were accustomed to using. Ditto the 5-year-old who corrected me when I asked what the man on the moon might see if he looked “down” at us? She blurted out, “he’d look up!,” setting my mind whirling on the concept of up and down!
Children aren’t being “cute” when they say surprisingly wise things; they are just being both observant and accustomed to taking their observations seriously. Too often, schooling cures them of both. In contrast, while schooling based on both these qualities allows us to cover less in the here and now, it rests on the conviction that—in the long run—it will cover more. Yes, “less” can just be “less”, contrary to one of my favorite Coalition of Essential Schools principles. It all depends on…..
And that’s where the craft of teaching begins—with “it all depends on…”
Deborah
A P.S. for future conversation. You and I remain unpersuaded by Murray and Herrnstein, Diane. I’m glad. But that a public bombarded with the apparent intractability of the “achievement gap” might fall back on centuries of racist/classist conclusions seems inevitable.
It’s one of the potential unintended consequences of the well-intended battle cry to “close the gap” without more carefully examining what test scores as we know them can and can’t tell us, and what other gaps we might start closing.