As more and more parents choose to opt their children out of standardized tests, some educators and teachers’ union representatives have been speculating about how all those missing scores might impact teacher-evaluation outcomes that are based on test results. According to a new analysis, the answer is: Probably not much.
Matthew Chingos and Katharine Lindquist of the Brookings Institute’s Brown Center on Education used past testing data to model the effects of opt-outs on New York teachers’ evaluations. They ran the numbers to determine an elementary school teacher’s chances of being rated ineffective based on the number of students who opt out, as well as the effect of only high-performing students doing so.
In general, the researchers found that “opting out adds noise to the data” but would likely have only a “trivial impact” on evaluation rates in most scenarios.
“A teacher faces a higher risk of being labelled low-performing (or high-performing) as the number of opt-outs in her classroom increases. But the effect of opt-out is quite small unless a large number of students do so,” Chingos explains in his paper on the study.
The researchers found that with only one student opting out, a teacher would have a seven percent chance of being rated ineffective—the same as if no students had opted-out. With ten randomly-selected students opting out, the teacher’s chances of being rated ineffective shifts just barely, to eight percent. It would take 15 high-performing students sitting the tests out—a majority of the class—to bump the teacher’s chances of being rated ineffective to 13 percent.
By the same token, with fewer students taking the test, the chance of teachers being rated highly effective on the basis of their value-added score goes up as well. In that same scenario—with 15 high-performing students opting out—the probability of earning that label doubles, from seven percent to 14 percent.
The analysis looked only at 4th and 5th grade students and teachers; for higher grades, where teachers may have well over a hundred students, students would have to opt out in even higher numbers for there to be a noticeable effect on results.
Chingos and Lindquist used data from North Carolina, as those data were readily available to them. However, Chingos says that the findings should still apply in New York (or any other state with a similar evaluation system, for that matter). While the specific percentages involved might vary were they to use scores from another state, the overall result would likely be the same: It takes a fairly high number of opt-outs to make a difference.
To a degree, the Brookings analysis backs up what Education Week‘s Andrew Ujifusa heard recently from Derek Briggs, a professor at the University of Colorado, on the subject of opt-outs potential effects on school accountability systems. Briggs explained that while opt-outs in certain student groups could affect particular measurements, the test scores would still be statistically valid for most accountability purposes even with an overall participation rate of substantially less than the 95-percent level required by No Child Left Behind.
Update: The original version of this post misspelled the surname of Matthew Chingos in several instances.
A version of this news article first appeared in the Teaching Now blog.