Measuring Accountability When Trust Is Conditional
On a designated graffiti wall at the City University of New York Queens College, circa 1970, the word “accountability” never appeared. But the concept dominated the informal curriculum. Like most colleges then, Queens College was mired in the struggles of the day, and with our sit-ins and teach-ins and caravans to the nation’s capital, I believe we were indeed trying to hold our government accountable: for stagnation in the civil rights movement, for the ravages of poverty, for the disastrous war in Vietnam.
One of the more obvious entries on that mural, amid the colorful exhortations to “burn pot, not people,” “support our boys—bring them home,” “black is beautiful,” and “abolish required courses NOW!” was “CHALLENGE AUTHORITY,” as good an abbreviation of the Declaration of Independence as one could hope for with a paint sprayer. The phrase may seem banal, but unpacking it helps me understand today’s education policy debates. (The Chicago teachers’ strike was, at its core, a fight over authority. Just who was challenging whom, though, is complicated.)
To challenge authority is to hold authority accountable. Challenging people in power requires them to show that what they are doing is legitimate; we invite them to rise to the challenge and prove their case; and they, in turn, trust that the system will treat them fairly. Mutual trust is key, and sequence is important. We entrust our leaders with authority to tax us, send our men and women to war, keep our air and water clean, police our neighborhoods, protect us from foreign threats, school our children, and even, God willing, some day provide a decent system of health insurance. Then we demand evidence: Prior trust granted on the basis of political rhetoric evolves into trust in the reliability and appropriate use of valid indicators of performance.
All this may sound intuitive, but it is devilishly complex to implement. We spend large sums on the design and validation of measures to challenge authority—and then we challenge the authoritativeness of the information we get!
One of the wiser graffiti artists at Queens understood this who-sent-you side of our political culture and under “CHALLENGE AUTHORITY,” scrawled a crushing two-word rebuttal: “SAYS WHO?” I have always thought that this playful dialogue is a metaphor of what is great—and frustrating—about American education. Our problem is not a lack of opinion and expertise, but rather an oversupply. For every proposed reform, there arise skeptics whose first challenge is essentially: SAYS WHO?
Indeed, because of our allergy to almost any form of authority—political, scientific, religious—it took 214 years for the United States to draft a set of national educational goals and another 20 years to develop the common-core standards. The phrase “national curriculum” is still taboo. Visionaries who worry that our world standing will be eroded if children in Montana and Florida learn differently are locked in battle with visionaries who prefer locally grown agility to clogged centralized hierarchies. The possibility that truth lies somewhere in between doesn’t make for good headlines or campaign oratory.
Teachers have become the target of our toughest scrutiny. The basic proposition is that they have substantial authority in their classrooms, and that entrusting them with our children entitles us to evidence of their performance. Anxiety about the condition of education has been translated into rigorous but, to many observers, onerous and unfair efforts to measure and publicize teacher quality. Even if the metrics are new and the way teachers are at times subjected to public shame and ridicule is hideous, the basic idea is familiar. We have held teachers and schools accountable at least since the common-school reforms of the early 19th century.
What’s different today is the federal role, though it pays to remember that protecting the public from powerful interest groups, teachers included, is one reason we have government in the first place. And supporters of the federal role have history on their side: Memories of when local control meant states’ rights applied to the denial of civil rights are still fresh and raw. No wonder reform and accountability are so complicated.
Another difference is perhaps more significant. Unlike the good old days when accountability meant challenging the ruling elites, the current reform strategy with its emphasis on teacher performance targets a professional class of people who are essentially employees of the state. The traditional model, in which students and workers rise up to criticize their leaders, has been inverted: Elected officials—in the Chicago case, Democrats with traditional loyalties to the labor movement—along with powerful corporate and foundation executives are accused of holding working teachers (and their students) accountable unfairly. For the sarcastically inclined among them, the graffiti shorthand would be simple: “
Workers, CEOs of the world unite.”
Humor aside, critics of this strategy of accountability rightly worry that the pressure undermines—rather than enhances—performance. Recent polling data suggest that teacher morale has hit an all-time low, and there is gobs of evidence linking productivity to worker satisfaction. Still, the argument that if we trusted our teachers, they would do a better job, misses the point: Accountability is all about conditional trust. We trust that teachers will work hard, but we reserve the right to evaluate them.
No profession is granted automatic autonomy or an exemption from evaluation. The challenge is where to set the dial between blind trust at one extreme and stifling control at the other, which is why the choice of metrics and the fairness of evaluation processes is so important. Just ask the teachers who went on strike in Chicago.
Americans are right to be suspicious of subjective criteria used capriciously to evaluate performance; and contrary to popular rhetoric, the science of measurement has resulted in more valid and fairer inferences derived from test scores. Still, the fact that scores are estimates seems to be lost on policymakers and pundits who rely on tests as though they were as accurate as, say, thermometers that measure temperature. Worse yet, attaching serious consequences to the results invites opportunistic mischief and further erosion of validity. As the estimates get cruder, the meaning of the results becomes blurred, and we lose trust; without trust, accountability is impossible; and without accountability, democracy is meaningless.
The eminent social scientist James March warned that “the demand for accountability is a sign of pathology in the social system.” If he were a graffiti artist, I suspect he would evoke President Ronald Reagan’s approach to the former Soviet Union and summarize our attitude to teachers with “trust” in small letters and “VERIFY” in big, bold strokes.
Nevertheless, I will close on a note of cautious optimism: Demand for accountability is not always a sign of pathology, but rather can be a sign of robust democracy. The fact that no accountability system will ever be perfect should not deter us from experimenting carefully with innovative metrics and evaluation designs, from investing in the restoration of trust in our teachers, from building public confidence in data, and from holding our accountability system ... accountable. These problems don’t have “optimal” solutions, and anyone claiming to have the quick fix should anticipate new graffiti. It will say “MISSION ACCOMPLISHED”—to which no written rebuttal will be needed.
Vol. 32, Issue 05, Pages 27, 32Published in Print: September 26, 2012, as Measuring Accountability When Trust Is Conditional