Accountability in public education requires a balance of audit and inspection. Audit assesses school performance, by comparing past results (usually student achievement-test scores) to current outcomes, or comparing schools to each other, or to a local or national standard. Inspection involves processes of observation and inquiry that analyze school performance and determine how it might be improved. Public education accountability in the United States has always emphasized audit. But the recent issuing of letter-grade report cards to every New York City public school is a triumph of audit that disserves all our public schools. (“N.Y.C. District Issues ‘Value Added’ Grades for Schools,” Nov. 14, 2006.)
At its most technical levels, the New York City accountability system is problematic. The city’s department of education has acknowledged substantial data error, inevitable when multiple large databases are aggregated to produce a single score for 1,200 individual schools. Inexplicably, the department used only two years of testing results for its comparisons, yet much research indicates that year-to-year school level fluctuation in test scores makes such comparisons meaningless. Sophisticated bloggers throughout the city have found multiple flaws in the department’s construction of similar school-comparison groups and its scoring intervals. The ability of prestigious city schools to get their grades changed suggests the political manipulability of the entire process.
But the city’s report card system is fundamentally flawed in conception as well as in execution. Audit—the measurement of schooling outcomes—cannot improve schools; it can only identify schools that are performing well or poorly. The theory behind audit systems is that when school performance is quantified, we can identify and reward the effective schools, punish the ineffective ones, and improvement in student achievement, presumably the goal of the audit effort, will result.
Audit—the measurement of schooling outcomes—cannot improve schools; it can only identify schools that are performing well or poorly.
But can school improvement result from audit efforts? If poorly performing schools knew how to improve their outcomes, surely they would do so rather than risk the public identification and shaming inherent in an audit. Suppose, instead, that limited capacity is the critical problem of poorly performing schools. Suppose they suffer from inexperienced leadership, high staff attrition that results in a large percentage of new teachers every year, a level of teaching skills too limited to effectively teach the curricula, or an inability to adapt instructional approaches to students’ learning needs. If these key capacity issues bedevil poorly performing schools, how will audit efforts that assign failing grades contribute to improving such schools?
Improving the outcomes of poorly performing schools requires building their capacity for more effective performance. Inspection processes help improve schools’ capacity because they analyze what schools are doing—or, more important, not doing—that produces poor student achievement. Once inspection identifies what poorly performing schools should do to improve, such schools can build their capacity for effective performance—if they are given the appropriate supports. Thus an inspection system, with sufficient support and assistance, can contribute to improving poorly performing schools. An audit system, like accounting, can only report good and bad news.
The ultimate irony is that New York City has developed an inspection system, the Quality Review process, that observes and analyzes the processes of teaching and learning in all district schools. But comparing the findings of these quality reviews with the schools’ report card grades yields conflicting outcomes. Many schools receiving positive quality reviews get D’s and F’s on their report cards, while many receiving A’s and B’s get quite critical quality reviews. The education department has not attempted to explain or adjust these disparate outcomes. Worse, the results of the inspection system are not factored into the report card grades.
Thus, the New York City Department of Education, at a cost of more than $80 million, has imposed what amounts to an elaborate accounting system on the city’s public schools. But accounting is not accountability; many schools will be irreparably harmed, with a few well served, by this formula-driven exercise. Furthermore, the department has, as usual, ignored history. The practice of letter-grade school report cards has been tried and discarded in many settings, Florida’s negative experience being only the most recent example.
Less than two years remain before the New York state legislature must reauthorize, revise, or terminate mayoral control of New York City’s public schools. One can hope that a citywide dialogue will begin, to debate the wisdom of unfettered control over public schooling that produces the kind of quantitative hubris that the report cards represent.
