Assessment Opinion

The Declining Value of Value-Added Models, and Why They Persist Anyway

By Dave Powell — March 02, 2015 7 min read
  • Save to favorites
  • Print

I’ve been riding the conservatives a little hard lately with my criticism of their crazy obsession with making laws to codify various forms of curriculum-oriented government overreach, so it’s time to turn some attention to others in need of questioning. I want to be an equal-opportunity contrarian, at least as far as writing about bad ideas is concerned.

You might have heard in January about the Obama administration’s abrupt about-face on taxing 529 programs. This was a startling change in policy brought on by nothing more than political pressure being applied by the right people at just the right pressure points. The plan’s demise came about five minutes after it was announced—pretty fast, even for Washington pull-the-bad-idea-off-the-table-as-quickly-as-possible-before-too-many-stories-get-written-about-it standards.

What you didn’t hear about was the Obama administration’s decision to finally abandon the absurdity known as “value-added” assessment as a way of determining whether or not teachers are effective, because it didn’t happen. This leads to one of two conclusions: either the administration still just doesn’t know about the shortcomings of these methods, or those of us who see those shortcomings aren’t applying the right kind of pressure at the right pressure points to get the administration to see the light. Maybe both.

One thing’s for certain: the administration has not changed its mind. Just recently it was soliciting comments on a proposal to extend the reach of value-added models into teacher preparation programs—a topic for another post. I happen to live and work in a state, Pennsylvania, that is piloting the use of these models in K-12 settings right now. Here we call it PVAAS, which stands for “Pennsylvania Value-Added Assessment System,” and the state has invested an untold amount of money and man-hours in making sure that teachers know that the framework will be implemented to evaluate their effectiveness. We’re all in. My brain’s not big enough to even begin to want to try to figure out exactly what’s going on here, but I know enough to know this: the program is another triumph of bad policy that gets us even further away from actually evaluating what makes effective teachers effective.

The problems here are so numerous that they can’t all be discussed at once. For example: a leading professional organization, the American Statistical Association, questions the validity of value-added methodologies a as means of assessing the effectiveness of teachers. “Ranking teachers by their VAM scores can have unintended consequences that reduce quality,” is how they put it, somewhat dryly. We also know that people respond to incentives, and we know that people are smart enough to know that it’s harder to “add value” to a student’s academic experience when, say, the student shows up to school hungry, or goes home from school to an unsupervised environment where books and quality time with adults may not be easily accessible. It’s not hard to imagine that teachers—especially young teachers—might not want to accept teaching jobs in places where students face so many challenges outside of school if those teachers know their ability to continue earning a paycheck is going to be contingent on their ability to teach kids how to do things like read and do math all by themselves with very little support. (And, make no mistake about it: teaching is something you have to learn how to do. It’s the rare first-year teacher who is able to be as effective as even some mediocre teachers who have been in the classroom for 7-10 years.)

Of course, the whole enterprise only works if there’s a steady supply of good teachers to take the places of the ones deemed ineffective by the evaluation system. That’s the problem with these short-sighted “solutions” to complex problems: by treating the symptom and not the disease, they actually expose us to the risk of making things much worse. What happens when the supply of teachers dries up? Because, again, if you think college graduates will choose to “explore teaching” once the economy actually improves, and once they realize that teaching has been reduced to doing what you’re told, my guess is that our best students won’t be lining up to take the job. For as long as I’ve been a teacher I’ve been hearing about the looming teacher shortage. Fifteen years in, it’s still looming, apparently, and in the meantime we’ve weakened the rights of teachers, lowered their pay, increased their class sizes, made them teach to the test, questioned their intelligence, insulted them pretty much without rest, and done just about everything else we could think of to make teaching the least attractive job possible. This is our plan for addressing the coming shortage of teachers?

One logical conclusion to draw from the demonization of teachers is that this is all part of the plan: once the bad teachers are all flushed out, and the worst public schools are all closed, and the idea of choice is normalized in the minds of education “consumers,” we’ll have finally deregulated the last real public institution run by the government. It’s a logical conclusion, to be sure, but it’s a little too pat for me. I’m not paranoid enough to believe that there are puppetmasters coordinated enough to foist standardized tests and common core and VAM on us to ensure that their privatization scheme finally works.

But that doesn’t mean things aren’t coordinated circumstantially. This is often how policymaking works in our political process—and VAMs occupy a policymaking sweet spot. Daniel Patrick Moynihan once said that the central conservative truth is that culture, not politics, determines the success of society, and that the central liberal truth is that politics can change a culture and save it from itself. Applying these untested ideas to education satisfies both sides: conservatives can rest easy knowing that VAMs are changing our culture by holding teachers (a group that, especially when unionized, conveniently does not tend to support conservatives in the political process) “accountable” for meeting unrealistic goals. Change the culture and you change the politics. If the teachers fail, new teachers will come along to take their place. And if those teachers are less qualified and less effective, who cares? They’ll just reinforce the idea that public schools don’t work because they can’t work. Then what? Who knows. But we’ll burn that bridge, too, when we come to it.

On the other hand, using the political process—in this case, administrative or executive authority derived from electoral success—to force change on what’s perceived as a stagnant culture fits nicely with Moynihan’s description of the central liberal truth. Thus President Obama can charge full speed ahead on a reform proposal that helps him use his executive power to try to rid bad schools of bad teachers (yes, they exist), as he proposes to do through rulemaking aimed at teacher education programs. In state departments of education that have decided to lean on VAMs the only concern at all, apparently, is compliance with federal law, which is kind of puzzling but no less disastrous where education policy is concerned. VAMs promise to streamline all the processes associated with meeting AYP, even if AYP might be on its way out, making reporting easier. And, as they say, states are the “laboratories of democracy"—or perhaps the laboratories of questionable, narrow-minded, mass-distributed ideas that would never see the light of day if most people knew anything about them. Anyway, as Moynihan pointed out, no good liberal can resist using the political process to try to readjust our cultural settings, and that’s what seems to be happening here.

The only thing left to do after the reform has been chosen is manufacture consent from the public to support it, and the best way to do that may be to make sure the public doesn’t really know or understand what’s actually happening. Obama has come in for his share of criticism from the left for abandoning traditional Democratic priorities where education is concerned, and we know how the right feels about him. But he’s shrewd. He knows how to use leverage and apparently believes in what he’s doing. He used his political capital wisely just after being elected to make significant changes in the education system. Conservatives, meanwhile, have benefited from Obama’s largesse: as his activism is inevitably defined as overreach by his nervous opponents, their state-level machinations promoting choice have come closer to fruition. Like VAMs, charter schools also check off the right boxes—conservatives see them as organic cultural game-changers, and (neo-)liberals see them as rational structural solutions to an old problem. And standardized tests reframed as a civil right that can guarantee that no child gets left behind while holding teachers and schools accountable? That too. You can see why things work the way they do. Who says these two sides can never agree on anything?

Related Tags:

The opinions expressed in The K-12 Contrarian are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.