Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
Teaching Profession

A Look at the Fuss Over Value-Added Teacher Data

By Barbara Stewart & The Hechinger Report — October 22, 2010 6 min read

Good teachers matter and—as in every other profession—some are better than others. Researchers have even found that the very best teachers can help students overcome many of the effects of poverty and catch up to or surpass their more privileged peers.

That’s why there is intense interest now in finding better ways to judge the relative effectiveness of teachers. But how should that be done? Most teacher evaluations not only fail to single out successful teachers—they also don’t help principals determine which teachers need help to improve and which ones are failing their students altogether. Instead, all teachers end up being judged the same, which is to say, satisfactory.

“It’s universally acknowledged—teacher evaluations are broken,” said Timothy Daly, president of The New Teacher Project, a group that helps school districts recruit and train teachers.

Perhaps surprisingly, teacher-union leaders agree. Michael Mulgrew, president of New York City’s United Federation of Teachers (UFT), said last spring that “the current evaluation system doesn’t work for teachers—it’s too subjective, lacks specific criteria and is too dependent on the whims and prejudices of principals.”

So, it would seem that a system using student test scores to calculate how much “value” teachers add to their students’ learning would be fairer. Indeed, Mulgrew endorsed New York state’s new evaluation system, in which student achievement counts for 40 percent of a teacher’s rating.

“Value-added” measurements use complex statistical models to project a student’s future gains based on his or her past performance, taking into account how similar students perform. The idea is that good teachers add value by helping students progress further than expected, and bad teachers subtract value by slowing their students down.

Using value-added models to calculate teacher effectiveness wasn’t possible on a wide scale until recently. In the 1990s, William L. Sanders, a statistician at the University of Tennessee, pioneered the technique with student test scores—and managed to persuade the Hamilton County School Board to work with him in taking a closer look at the results.

The method—as Sanders puts it—is like measuring a child’s height on a wall. It tracks a child’s academic growth over the year, no matter how far ahead or behind the child was initially. Sanders discovered that teacher quality varied greatly in every school, and he and others also found that students assigned to good teachers for three consecutive years tended to make great strides, while those assigned to three poor ones in a row usually fell way behind.

Why Value-Added Is Hot Now

Hundreds of districts, including Chicago, Denver, New York City and Washington, D.C., are using such methods as a way to strengthen their teacher evaluations by factoring in student performance. The biggest push for the use of such methods has come from the Obama administration, which insisted that states competing for grants under its $4.3 billion Race to the Top program find ways to link student performance to teacher evaluations. The dozen winners of those grants are now struggling to figure out how to do just that.

At the same time, however, value-added modeling is the focus of furious debate among scholars, policymakers, superintendents, education advocates and journalists. The latest flare-up is occurring this week in New York City. The New York Times, Wall Street Journal, New York Daily News and others are seeking the value-added rankings of about 12,000 teachers in grades 4 through 8 whose students took state English and math tests.

The New York City Department of Education says it’s willing to make those scores public. But the UFT is suing to block their release. Value-added ratings, the union says in its lawsuit, are “unreliable, often incorrect, subjective analyses dressed up as scientific fact.” The union calls the calculations a “complex and largely subjective guessing game.”

In August, the Los Angeles Times was the subject of intense criticism and praise for its series that included value-added scores for individual teachers based on years of standardized test data—a project that newspapers in New York City now want to replicate. (Disclosure: The Los Angeles Times data-analysis was supported in part by a grant from The Hechinger Report.)

The documentary “Waiting for ‘Superman’,” directed by Davis Guggenheim, also has thrust the teacher-evaluation issue into the national spotlight, highlighting as it does the historical disconnect between teacher job-security and student performance.

Limitations of Value-Added

Value-added models aren’t perfect, as even their most ardent supporters concede. Oft-cited shortcomings range from doubts about fairness to broader concerns centered on teaching goals. “When people talk about their experience with a really good teacher, they’re not talking about test scores,” said Aaron Pallas, professor of sociology and education at Teachers College, Columbia University. “They’re talking about a teacher who gave them self-confidence, the ability to learn, an interest and curiosity about certain subjects.”

Critics point out that value-added data are only as good as the standardized tests—and test quality varies greatly from state to state. There are also many ways to calculate value-added scores, and different statistical techniques yield different results. The calculations may take into account factors that can affect achievement, such as class size, a school’s funding level and student demographics. Whether to include the race and poverty-status of students when measuring teachers is particularly contentious, writes Douglas N. Harris, an economist at the University of Wisconsin, Madison, in a report on value-added models released this week.

Whatever the computational method, a teacher’s score can vary significantly from one year to the next—results that could affect a teacher’s reputation and salary in places that are considering linking teacher pay to performance. And while value-added models may do a decent job of highlighting the best and worst performers, they’re widely considered unreliable in differentiating the good from the mediocre (or the mediocre from the terrible).

For this reason, many want value-added calculations only to be used in assessing schools and curricula—not individual teachers. But more and more, value-added data are playing a role in personnel decisions about bonuses, tenure and dismissal. In July, D.C. Schools Chancellor Michelle Rhee made waves by firing 165 teachers for poor evaluations, half of which depended on value-added data. With Adrian Fenty’s loss in the Democratic primary for mayor of D.C. last month, Rhee has announced her resignation—but her teacher-evaluation system, IMPACT, will remain in place.

What Lies Ahead

Even those who champion value-added measures caution against using them as the sole means of evaluating teachers. Kate Walsh, president of the National Council on Teacher Quality, a research and advocacy group in Washington, D.C., has called value-added the best teacher-evaluation method so far. But she also says it would be a “huge mistake” to rely on it alone, or even primarily.

Randi Weingarten, president of the American Federation of Teachers, does not oppose the use of value-added data but wants to ensure evaluations are based on “classroom observations, self-evaluations, portfolios, appraisal of lesson plans, students’ written work” as well.

The best uses of value-added data may well be in the future. If educators could use the data to figure out what the most effective teachers are doing right and share that with colleagues, it would be a great boon. But while major foundation money is being spent to try and do just that, it is very difficult, especially given that great teachers often don’t know themselves what they’re doing right.

Whatever the future uses of value-added measures, the idea of holding teachers accountable for student performance seems here to stay.

“It’s a valuable part of the conversation,” said Daly, of The New Teacher Project. “It puts what matters most—student achievement—front and center as the most important responsibility for a teacher.”

The Hechinger Report’s Sarah Garland and Richard Lee Colvin contributed to this article. The Hechinger Report is a nonprofit, nonpartisan education news outlet affiliated with the Hechinger Institute on Education and the Media, based at Teachers College, Columbia University.
A version of this article appeared in the October 27, 2010 edition of Education Week

Events

Teaching Live Online Discussion A Seat at the Table With Education Week: How Educators Can Respond to a Post-Truth Era
How do educators break through the noise of disinformation to teach lessons grounded in objective truth? Join to find out.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
The 4 Biggest Challenges of MTSS During Remote Learning: How Districts Are Adapting
Leaders share ways they have overcome the biggest obstacles of adapting a MTSS or RTI framework in a hybrid or remote learning environment.
Content provided by Panorama Education
Student Well-Being Online Summit Keeping Students and Teachers Motivated and Engaged
Join experts to learn how to address teacher morale, identify students with low engagement, and share what is working in remote learning.

EdWeek Top School Jobs

Join us for our NBOE 2021 Winter Teacher Virtual Interview Fair!
Newark, New Jersey
Newark Public Schools
Special Education Teacher
Chicago, Illinois
JCFS Chicago
Assistant Director of Technical Solutions
Working from home
EdGems Math LLC

Read Next

Teaching Profession Here Are the Four Finalists for 2021 National Teacher of the Year
The four finalists for the 2021 National Teacher of the Year challenge inequities both in their school communities and on a national level.
5 min read
From left: John Arthur (Utah), Alejandro Diasgranados (D.C.), Maureen Stover (N.C.), Juliana Urtubey (Nev.)
From left: John Arthur (Utah), Alejandro Diasgranados (D.C.), Maureen Stover (N.C.), Juliana Urtubey (Nev.)
Courtesy of CCSSO
Teaching Profession Has the Public Turned on Teachers?
The rift over when school buildings should reopen has caused teachers to feel like they've lost public support. But have they?
12 min read
Cyndi Pristello, a school wide support teacher at JoAnna Connell Elementary School, greets students and their families on April 18, 2020, as they drive through the school's parking lot during a drive-by parade. The school will begin online classes on April 20 after being closed due to the coronavirus shutdown.
A teacher greets students and families during a drive-by parade in Erie, Penn. Many schools around the country remain closed, and some teachers say they're taking heat for it.
Jack Hanrahan/Erie Times-News via AP
Teaching Profession Chicago Teachers Vote to Continue Remote Teaching
Chicago Teachers Union announced Sunday its members "overwhelmingly" chose to conduct only remote work beginning Monday.
Katherine Rosenberg-Douglas & Bill Ruthhart
4 min read
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Anthony Vazquez/Chicago Sun-Times via AP
Teaching Profession After a Stillbirth, This Teacher Was Denied Paid Leave for Recovery. Here's Her Story
A District of Columbia teacher delivered a stillborn baby and was denied paid maternity leave. Her story, told here, is not uncommon.
6 min read
Illustration of a woman.
iStock/Getty