There was a bit of a mini-controversy in June when the New Teacher Project released its Widget Effect report.
But it wasn’t the report’s overall thrust that did it. Pretty much everyone agreed that our current systems for evaluating and offering assistance to struggling teachers are crummy.
The controversy was about the data on dismissals in one particular district: Toledo, Ohio. According to the district’s personnel records, Toledo dismissed one tenured veteran and did not renew five novice teachers’ contracts, the NTP reported.
But what about the district’s much-heralded Peer-Assistance and -Review model, a number of sources wrote me afterward. Aren’t the dismissal numbers much higher than that? Most people were merely confused, but some accused the NTP of willfully skewing the data.
And the American Federation of Teachers put out this press release:
While the overarching conclusions of the report are sound, we have concerns about the report's data, particularly with respect to teacher evaluations in Toledo, Ohio. Toledo has a highly regarded teacher evaluation system ... that produces much better results than those described in this report."
So what gives? Well, I’ve been talking to folks on both ends and I’ve started to do some digging around the numbers. To put it kindly, the data-gathering is really a mess. What it boils down to is that the district and the Toledo Federation of Teachers had entirely different ways of “coding” dismissals through the PAR system.
For instance, the district doesn’t appear to record teachers who resigned after being in the program as having been dismissed, but the union counts them as such. Also, it appears that the PAR program applied to some long-term substitute teachers. These teachers wouldn’t have necessarily shown up in personnel records, but the union may have counted them within the overall PAR figures. And finally, the union used the term “terminated” teachers to refer both to nontenured teachers who were so poor that they were let go before their contracts were up, and to tenured veterans who were put into PAR and ultimately dismissed. The district, by contrast, separated out tenured from non-tenured teachers.
There’s a lesson here for any district or union that wants to try peer review: Agree on common definitions and change your data systems accordingly, or it will be really hard to justify your numbers.
OK, I know you want the actual figures. Well, I’m not going to publish what I’ve got for a couple of reasons. First, my information isn’t complete, and second, it’s my understanding that officials from TFT and NTP are working together on trying to audit the numbers and come up with figures they can agree on. I don’t want to insert myself in that process.
But count on the fact that we’ll be bringing you updates once we know what’s what.
Does the Toledo PAR really produce “much better results” than other evaluation systems, as AFT has asserted? We’ll see.
A version of this news article first appeared in the Teacher Beat blog.