You have to hand it to the New York City Department of Education’s Department of Assessment and Accountability. You really do.
Yesterday morning, the NY Times reported that the DOE will now distribute teacher value-added reports to teachers and principals.* Here’s the thing - the value-added reports don’t just report that a teacher performs at the 65th percentile, the 25th percentile, etc. Instead - as they should - the DOE reports a confidence interval around each teacher’s value-added to represent the uncertainty of the estimate. And unsurprisingly, these confidence intervals are quite wide. A 65th percentile teacher in this example has a confidence interval ranging from the 46th to the 84th percentile. In providing this range, the DOE is formally acknowledging that we do not know if this is a below average, average, or above average teacher.
So imagine my surprise when DOE Accountability Czar Jim Liebman popped over last night to criticize skoolboy’s post showing that the report cards are flawed because they don’t address the uncertainty of the estimates of school progress. To Liebman, this is so much sound and fury.
Morning Jim Liebman, it’s my pleasure to introduce you to Evening Jim Liebman. You two are going to be BFF. Or frenemies. It’s hard to tell. Though you are the same person, you have remarkably conflicting views about what role uncertainty should play in accountability reports. Maybe it’s the coffee.
What beats me is why the DOE reports confidence intervals in their teacher reports if uncertainty is just a concern of pesky bloggers like eduwonkette and skoolboy. By providing confidence intervals on the new teacher reports, the DOE basically concedes that their school grading system is bunk. Oops!
* Sidenote: I question the wisdom of giving teachers and principals information that is likely to be inaccurate, based on all of the reasons articulated here (in short, New York’s testing schedule is problematic), but that is for another post.