Over at The Quick and the Ed, one of the many house organs of Education Sector, Kevin Carey is conducting a serial monologue belittling eduwonkette as an “alleged social scientist.” “Alleged”? Yeah, I’ll allege it – eduwonkette is a social scientist. It’s not an epithet, as much as Carey might believe; to some of us, it’s a way of life.
What’s the latest bee in Carey’s bonnet? It’s eduwonkette’s contention that particular value-added assessment systems for evaluating teacher performance are not ready for prime time. Carey views the claim that a particular policy alternative has identifiable flaws as tantamount to embracing a status quo that is demonstrably flawed. Public education clearly isn’t working, he argues. Therefore, any policy alternative to the status quo is to be preferred. Anyone who raises caveats about any kind of change is just an apologist for the status quo, a weasel, and probably a bed-wetter too.
The problem, Carey opines, is that social scientists such as eduwonkette – wait a minute, is she a social scientist or not? – have unrealistic standards for evaluating policy alternatives. “The standard in public policy isn’t 95%,” he writes, “ it’s whatever is most likely to be best: 51%. “ I’m not sure what the 95% refers to here, but most policy analysts I know are in the business of trying to recommend a policy alternative based on multiple criteria: the likely consequences of the alternative for various desirable outcomes; its cost; its feasibility and sustainability; its consistency with public values; and the likelihood of successful implementation. The hard reality is that there often isn’t a very strong evidence base for making these judgments, and policy analysts have to consider a range of possible outcomes along these criteria (a confidence interval that expresses the uncertainty about what might happen), and confront the tradeoffs, because invariably no single policy alternative looks best across all of these criteria. Simply having a good big idea—choice, accountability, charters, vouchers, whatever—isn’t enough to carry the day, because the devil of public policy is in the details. The world of policy analysis is littered with examples of good ideas that were implemented poorly, and thus did not have the desired effects—even though they were very costly initiatives.
For this reason, scholars of policy analysis (e.g., Eugene Bardach of the Goldman School of Public Policy at UC-Berkeley) almost always recommend considering allowing present trends to continue undisturbed as one of a set of policy alternatives intended to address a problem condition. Enacting an alternative that costs more than the current approach and doesn’t work is arguably worse than the status quo.
As for value-added assessment systems for evaluating teacher performance, we need to consider particular policy alternatives to the status quo in particular settings, not the big idea of value-added assessment for evaluating teacher performance (which both eduwonkette and I agree is promising.) If I can return to the New York City case which eduwonkette has discussed at length, the one new issue with regard to policy analysis that I’d like to introduce is feasibility and sustainability. It’s my opinion—and I’m not a lawyer, just an alleged social scientist—that the New York City approach, which defines 50% of a particular teacher’s effectiveness on the basis of how that teacher’s students do in other teachers’ classes over which the teacher has no control, would not survive a legal challenge. Other policy analysts might disagree, and might therefore be more favorably disposed towards this particular alternative. Either way, though, good policy analysis considers feasibility and sustainability as important criteria in evaluating policy alternatives.