The new school year will see the value-added model playing an inordinate role in evaluating teacher effectiveness. In some states, the metric will count as much as half of a teacher’s rating, according to the Institute for a Competitive Workforce. The weight alone is cause for deep concern, but it is not the only troublesome factor, for reasons that are given short shrift in the debate.
I was reminded of this flaw after reading an obituary in The New York Times of Paul Meier, a prominent medical statistician who was one of the first and most vocal proponents of randomization. Prior to this protocol, the science of clinical trials in medicine lacked precision. It was common practice for researchers to give a new treatment to patients who they thought might benefit, and then compare the results with those of previous patients who were not treated. This strategy often introduced serious bias that led to false conclusions.
Meier insisted that researchers randomly assign one group to receive an experimental treatment and another to receive the standard treatment. This prevented the unintentional distortion of results by the selection of healthier or younger patients to receive the new treatment. In other words, the two groups being compared would be the same in all respects except for the treatment they received. As a result of Meier’s work, randomized controlled trials became the gold standard for gathering evidence.
What does this have to do with teacher evaluation? The fact is that the overwhelming majority of schools do not randomly assign students to teachers. This policy caused little concern in the past because teachers were not evaluated on the basis of the value they added to their students’ knowledge and skills. But as Jesse Rothstein wrote: “Non-random assignment of students to teachers can bias value added estimates of teachers’ causal effects” (“Student Sorting and Bias in Value Added Estimation: Selection on Observables and Unobservables”).
States can’t have it both ways. If they are going to adopt the value-added model as the centerpiece of their teacher evaluation systems, then it behooves them to use random assignment of students. By failing to do so, they seriously call into question the inferences made about the effectiveness of individual teachers. Confidence is indispensable when the stakes are so high.