Two bills in the Tennessee legislature would eliminate the nation’s “best example” of using students’ learning gains to evaluate the effectiveness of their teachers.
“The Real Value of Teachers: Using New Information about Teacher Effectiveness to Close the Achievement Gap,” is available from the The Education Trust. (Requires Adobe’s Acrobat Reader.)
Sen. Thelma Harper and Rep. Mike Turner, both Democrats from Nashville, introduced the measures to do away with the Tennessee Value-Added Assessment System.
Enacted in the early 1990s, the system measures the growth that students make from the beginning to the end of the school year, based on standardized tests. Teachers and schools are rated on whether their students make more or less progress than a typical student is expected to make in a given subject and grade, after adjusting for the prior achievement of each child.
A report released last week by the Washington-based Education Trust argues that value-added measures offer the best way to identify and reward effective teachers and to get more such teachers into the classrooms of poor and minority students.
“Easily the best example is the system that’s currently up and running in Tennessee,” declares the report, “The Real Value of Teachers: Using New Information About Teacher Effectiveness to Close the Achievement Gap.”
Research in Tennessee has found that, all else being equal, students assigned to the most-effective teachers for three years in a row performed 50 percentile points higher on tests, on a 100-point scale, than did similar students assigned to the least effective teachers.
But at least some in Tennessee question the reliability and value of the much-recognized program, which is used to assign schools letter grades on state report cards and to help evaluate teachers in some districts. Teachers also can opt to use the results to prove that they are “highly qualified” under the federal No Child Left Behind Act.
Concerns Mount
Much of the frustration does not center on the value-added method itself, but on a step its creator, William L. Sanders, takes before calculating the scores. Essentially, he tries to determine if the scale used to weigh the easiness or difficulty of individual test items is equivalent from year to year, so that test results are comparable from one year to the next.
That’s important, because the value-added system averages scores over three years. If the scale is inconsistent, Mr. Sanders makes an adjustment.
Concerns mounted starting in the fall of 2002, when the 71,000-student Nashville public schools noted significant discrepancies between what the raw test results showed and what the value-added measures showed, after Mr. Sanders had made his adjustments, particularly in 4th grade reading and language arts.
“When we looked at raw scale-score growth over time, we saw differences of 40 percent or more between his estimates and our unadjusted estimates,” said Paul S. Changas, the coordinator of student assessment for the district. Other grades and subjects also showed significant discrepancies.
As a result of Mr. Sanders’ adjustments, the value-added findings for some schools dropped precipitously, Mr. Changas said. In 2003, 70 percent of elementary schools in the state received failing grades for language arts on state report cards, and 33 percent received failing grades in reading, based on the value-added calculations. Those discrepancies have called the larger system into question.
Mr. Sanders, who previously worked at the University of Tennessee and now operates out of SASinSchool, part of the Cary, N.C.-based SAS Institute Software Co., said that his contract with the Tennessee education department requires him to make such adjustments when the test scales are not consistent from year to year. There’s no problem with the value-added methodology, he said.
“I don’t care what kind of accountability system you have,” he maintained, “when you have districts that do not look as good as local leadership would like to think, then they’re not going to feel as good about it.”
Bruce Opie, the legislative liaison for the state education department, said the agency was working with both sides in the debate “on how we might more appropriately use the system and maybe address the concerns of the bills’ sponsors.”
“The department feels like the value-added system is a valuable system, and it gives us some diagnostic data that we would not have any other way,” he said.
Mr. Changas said the Nashville district does not necessarily want to do away with the system, but it does believe adjustments are needed. Among other changes, district officials would like the state to develop multiple test forms at one time and reuse more test items from year to year, so that scales are more consistent. They would also like target gains to be based on actual longitudinal data for Tennessee students.
‘Terrible Loss’
While Tennessee debates whether to hold on to its system, the Education Trust wants more states to move in that direction.
With the advent of information technology, academic standards, and the annual testing required under federal law, the research and advocacy organization argues, more states can now measure the effectiveness of individual teachers in helping students learn.
Filling that “information void,” argues author Kevin Carey, a senior policy analyst at the trust, could help close achievement gaps between students of different racial and economic backgrounds, if states and districts agree to take a number of steps.
Those include giving schools and districts the information they need to identify effective teachers; paring regulations that prevent them from hiring and deploying such teachers where they are needed most; and providing more money to high-poverty, high-minority districts so they can compete in the marketplace.
States and districts also could use the information to pay effective teachers more, improve professional development and teacher preparation, and adjust day-to-day instruction.
“For us, value-added has been a powerful force for improving our schools,” said Dan Challener, the president of the Public Education Foundation in Chattanooga, Tenn. It has used the data to identify highly effective teachers.
Tennessee’s 41,000-student Hamilton County school system, which includes Chattanooga, also has used the data to attract the most effective teachers to struggling elementary schools. Teachers who consistently showed the highest test-score gains were guaranteed an extra $5,000 per year for three years if they taught in one of nine low-performing schools. Local businesses also provided the teachers with housing benefits and free graduate education.
All nine schools in the program have shown significant gains on state tests in each of the past two years, improving faster than other schools in the district.
If value-added assessment went away, Mr. Challener said, “I would say categorically it would be a terrible loss to the state.”
