Opinion
IT Infrastructure & Management Opinion

Schooling Beyond Measure

By Alfie Kohn — September 18, 2012 6 min read
  • Save to favorites
  • Print

The reason that standardized-test results tend to be so uninformative and misleading is closely related to the reason that these tests are so popular in the first place. That, in turn, is connected to our attraction to—and the trouble with—grades, rubrics, and various practices commended to us as “data based.”

The common denominator? Our culture’s worshipful regard for numbers. Roger Jones, a physicist, called it “the heart of our modern idolatry ... the belief that the quantitative description of things is paramount and even complete in itself.”

Quantification can be entertaining, of course. Readers love Top 10 lists, and our favorite parts of the news are those with numerical components: sports, business, and weather. There’s something comforting about the simplicity of specificity. As the educator Selma Wassermann observed, “Numbers help to relieve the frustrations of the unknown.” If those numbers are getting larger over time, we figure we must be making progress. Anything that resists being reduced to numerical terms, by contrast, seems vaguely suspicious, or at least suspiciously vague.

In calling this sensibility into question, I’m not denying that there’s a place for quantification. Rather, I’m pointing out that it doesn’t always seem to know its place. If the question is “How tall is he?,” “6 foot 2" is a more useful answer than “pretty damn tall.” But what if the question were “Is that a good city to live in?” or “How does she feel about her sister?” or “Would you rather have your child in this teacher’s classroom or that one’s?”

To be overly enamored by numbers is to be vulnerable to their misuse."

The habit of looking for numerical answers to just about any question can probably be traced back to overlapping academic traditions like behaviorism and scientism (the belief that all true knowledge is scientific), as well as the arrogance of economists or statisticians who think their methods can be applied to everything in life. The resulting overreliance on numbers is, ironically, based more on faith than on reason. And the results can be disturbing.

In education, the question “How do we assess kids/teachers/schools?” has morphed over the years into “How do we measure ... ?” We’ve forgotten that assessment doesn’t require measurement, and, moreover, that the most valuable forms of assessment are often qualitative (say, a narrative account of a child’s progress by an observant teacher who knows the child well), rather than quantitative (a standardized-test score). Yet the former may well be brushed aside in favor of the latter by people who don’t even bother to ask what was on the test. It’s a number, so we sit up and pay attention. Over time, the more data we accumulate, the less we really know.

You’ve heard it said that tests and other measures are, like technology, merely neutral tools, and all that matters is what we do with the information. Baloney. The measure affects that which is measured. Indeed, the fact that we chose to measure in the first place carries causal weight. His speechwriters had President George W. Bush proclaim, “Measurement is the cornerstone of learning.” What they should have written was “Measurement is the cornerstone of the kind of learning that lends itself to being measured.”

One example: It’s easier to score a student writer’s proficiency with sentence structure than her proficiency at evoking excitement in a reader. Thus, the introduction of a scoring device like a rubric will likely lead to more emphasis on teaching mechanics. Either that, or the notion of “evocative” writing will be flattened into something that can be expressed as a numerical rating. Objectivity has a way of objectifying. Pretty soon the question of what our whole education system ought to be doing gives way to the question of which educational goals are easiest to measure.

BRIC ARCHIVE

I’ll say it again: Quantification does have a role to play. We need to be able to count how many kids are in each class if we want to know the effects of class size. But the effects of class size on what? Will we look only at test scores, ignoring outcomes such as students’ enthusiasm about learning or their experience of the classroom as a caring community?

Too much is lost to us—or warped—as a result of our love affair with numbers. And there are other casualties as well:

We miss the forest while counting the trees. Rigorous ratings of how well something is being done tend to distract us from asking whether that activity is sensible or ethical. Dubious cultural values and belief systems are often camouflaged by numerical precision, sometimes out to several decimal places. Stephen Jay Gould, in his book The Mismeasure of Man, provided ample evidence that meretricious findings are often produced by impressively meticulous quantifiers.

We become obsessed with winning. An infatuation with numbers not only emerges from but also exacerbates our cultural addiction to competition. It’s easier to know how many others we’ve beaten, and by how much, if achievements have been quantified. But once they’re quantified, it’s tempting for us to spend our time comparing and ranking, trying to triumph over one another rather than cooperating.

We deny our subjectivity. Sometimes the exclusion of what’s hard to quantify is rationalized on the grounds that it’s “merely subjective.” But subjectivity isn’t purged by relying on numbers; it’s just driven underground, yielding the appearance of objectivity. An “86" at the top of a paper is steeped in the teacher’s subjective criteria just as much as his comments about that paper. Even a score on a math quiz isn’t “objective": It reflects the teacher’s choices about how many and what type of questions to include, how difficult they should be, how much each answer will count, and so on. Ditto for standardized tests, except the people making those choices are distant and invisible.

Subjectivity isn’t a bad thing; it’s about judgment, which is a marvelous human capacity that, in the plural, supplies the lifeblood of a democratic society. What’s bad is the use of numbers to pretend that we’ve eliminated it.

Skepticism about—and denial of—judgment in general is compounded these days by an institutionalized distrust of teachers’ judgments. Hence the tidal wave of standardized testing in the name of “accountability.” Part of the point is to bypass the teachers and indeed to evaluate them, too. The exalted status of numerical data also helps explain why teachers are increasingly being trained rather than educated.

To be overly enamored of numbers is to be vulnerable to their misuse, a timely example being the pseudoscience of “value-added modeling” of test data, debunked by experts but continuing to sucker the credulous. The trouble, however, isn’t limited to lying with statistics. None of these problems with quantification disappears when no dishonesty or incompetence is involved. Likewise, better measurements or more thoughtful criteria for rating aren’t sufficient.

At the surface, yes, we’re obliged to do something about bad tests and poorly designed rubrics and meaningless data. But what lies underneath is an irrational attachment to tests, rubrics, and data, per se, or, more precisely, our penchant for reducing to numbers what is distorted by that very act.

Related Tags:

A version of this article appeared in the September 19, 2012 edition of Education Week as Schooling Beyond Measure

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Leadership in Education: Building Collaborative Teams and Driving Innovation
Learn strategies to build strong teams, foster innovation, & drive student success.
Content provided by Follett Learning
School & District Management K-12 Essentials Forum Principals, Lead Stronger in the New School Year
Join this free virtual event for a deep dive on the skills and motivation you need to put your best foot forward in the new year.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Modern Data Protection & Privacy in Education
Explore the modern landscape of data loss prevention in education and learn actionable strategies to protect sensitive data.
Content provided by  Symantec & Carahsoft

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

IT Infrastructure & Management Los Angeles Unified's AI Meltdown: 5 Ways Districts Can Avoid the Same Mistakes
The district didn't clearly define the problem it was trying to fix with AI, experts say. Instead, it bought into the hype.
10 min read
Close up of female hand holding smartphone with creative AI robot hologram with question mark in speech bubble on blue background. Chat GPT and failure concept.
Peshkov/iStock/Getty
IT Infrastructure & Management Aging Chromebooks End Up in the Landfill. Is There an Alternative?
Districts loaded up on devices during the pandemic. What becomes of them as they reach the end of their useful lives?
5 min read
Brandon Hernandez works on a puzzle on a tablet before it's his turn to practice reading at an after school program at the Vardaman Family Life Center in Vardaman Miss., on March 3, 2020.
Brandon Hernandez works on a puzzle on a tablet before it's his turn to practice reading at an after-school program at the Vardaman Family Life Center in Vardaman Miss., on March 3, 2020. Districts that acquired devices for every student for the first time during the pandemic are facing decisions about what to do at the end of the devices' useful life.
Thomas Wells/The Northeast Mississippi Daily Journal via AP
IT Infrastructure & Management Schools Can't Evaluate All Those Ed-Tech Products. Help Is on the Way
Many districts don't have the time or expertise to carefully evaluate the array of ed-tech tools on the market.
2 min read
PC tablet with cloud of application icons floating from off the screen.
iStock/Getty
IT Infrastructure & Management FCC Pilot Program to Help Schools Fight 'Real and Growing' Cyberattacks
School districts and libraries can soon seek new federal grants to protect against the cyberattacks.
4 min read
Dollar Sign Made of Circuit Board on Motherboard and CPU.
iStock/Getty