UPDATED
Cross-posted from the Teacher Beat blog
The union for District of Columbia public school teachers is demanding more information about errors made in calculating teachers’ evaluation scores for 2012-13, although the school system says that the errors affected a relatively small number of teachers, and that officials are working with the contractor responsible to correct the problem.
The system’s chief of human capital, Jason Kamras, said in a Dec. 23 statement that the calculation error made by Mathematica Policy Research affected only 44 teachers out of a workforce of 4,000 teachers. UPDATE: However, those 44 teachers represent 10 percent of all DCPS teachers who are evaluated on a value-added model that uses student performance on test scores, according to Melissa Salmanowitz, a spokeswoman for DCPS. In addition, as a result of the miscalculations, one teacher was fired, although Salmanowitz also confirmed that DCPS is attempting to reinstate that teacher. (The Washington Post first reported on the details about the teacher who was dismissed, as well as the share of teachers judged using a value-added model, in a Dec. 24 story.)
Mathematica designed the system’s value-added model, which operates as part of the District of Columbia’s IMPACT evaluation system, and continues to help the district measure teacher effectiveness within that system. It has worked with D.C. schools on teacher evaluations since 2009. The specific error dealt with how teachers’ “individual value-added” (IVA) scores were calculated for the 2012-13 year—a teacher’s IVA score constitutes 35 percent of his or her total evaluation score.
In total, 22 teachers received a lower rating than they should have, and 22 teachers received a higher rating than they should have, according to the school system.
“It is important to note that the impact of the error was small but nonetheless, any error is unacceptable,” Kamras said in the statement. “Teachers who should have received higher IMPACT scores will receive notification of their corrected (higher) scores and will receive all the benefits (such as bonuses) that go with those scores. Teachers who should have received lower scores will be held completely harmless.”
Before Kamras’ statement was issued, however, Washington Teachers Union President Elizabeth Davis told district schools Chancellor Kaya Henderson in a Dec. 22 letter about the errors: “Needless to say, I was deeply disturbed by this preliminary information as well as the time at which it was provided—the day before the winter break.”
Davis said in her letter that she wants her union to be provided all pertinent information related to the evaluation errors. She has a long list of demands for data and of questions she wants answered. For instance, Davis wants to see a list of all teachers affected by the errors; copies of all correspondence sent to teachers regarding the miscalculation of their scores; the impact of the miscalculation on each teacher; a full description of the Mathematica error, the cause of the error, and how and when the error was brought to the attention of school district officials; and the number of teacher scores that were affected.
Among the questions Davis wants answers to that Kamras didn’t address are:
- What are the specific groups of teachers affected by these or other errors? Elementary? Teachers with individual student test scores? Teachers in a specific ward of the city?
- Was the error the result of actions taken by one or more employees of the school system? If so, was the person or persons responsible for evaluating or reviewing teachers’ evaluations?
- Were personnel actions taken against any administrator or private contractor as a result of the problem?
A call to Mathematica wasn’t immediately returned. Since the school system has downplayed the scope of the problem, it will be interesting to see how many of the union’s questions district officials answer.