Education

Princeton Study Takes Aim at ‘Value-Added’ Measure

By Debra Viadero — June 23, 2009 2 min read
  • Save to favorites
  • Print

Merit pay for teachers is an idea that seems to be getting increasingly popular these days with politicians on both sides of the aisle. But if performance-pay plans are going to succeed, they require an evaluation system that is widely seen to be fair and accurate.

A lot of policy makers are pinning their hopes on “value-added” measures of student achievement to fill that bill. The thinking is that value-added models provide a fairer measure of teacher effectiveness because they track students’ year-to-year learning gains, rather than their absolute levels of achievement. That way, teachers are not getting undeserved blame for the learning deficits that students bring with them to the classroom or undue rewards for being blessed with a classroom of high achievers.

A forthcoming study, however, suggests policy makers might want to think twice before embracing value-added measures of teacher effectiveness. In a paper due to be published in February in the Quarterly Journal of Economics, Princeton University economist Jesse Rothstein uses some sophisticated modeling techniques to suggest that such techniques could be based on shaky assumptions.

Using student-testing data from North Carolina, Rothstein makes his case by developing a “falsification” test for value-added models. For example, he wondered, would the model show that 5th grade teachers have effects on their students’ test scores in 3rd and 4th grades? Since it’s impossible for students’ future teachers to cause their previous achievement outcomes, Rothstein reasons, there should be no such effects.

But in fact there were—and they were quite large. Rothstein says this happens because students are not randomly sorted into classrooms. A principal, for example, might assign a large number of students with behavior problems to a teacher who is known to have a way with problem students or parents of high achievers might lobby to get their child in a class with the “best” teacher. When that happens, though, it biases the results of value-added calculations.

If this study sounds familiar, it’s because it’s been circulating a while and gathering lots of buzz. I’m not yet sure why the finding that a 5th grade teacher seems to cause students’ 3rd and 4th grade achievement automatically implies that students were not randomly sorted, but I hope to figure that out. Look for a more detailed story from me soon in Education Week.

In the meantime, you should know about two other studies that offer some counterpoint to Rothstein’s findings. Thomas J. Kane and Douglas O. Staiger, for one, conducted a small experiment in Los Angeles public schools to see if value-added calculations would match the experimental results. They did. See their paper, “Are Teacher-Level Value-Added Estimates Biased?: An Experimental Validation of Non-Experimental Estimates.

A second study, a working paper by Cory Koedel and Julian R. Betts, suggests that the kinds of biases that Rothstein highlights in his paper can be overcome with more complex value-added models.

You can find a summary of Rothstein’s paper and the full text on Princeton’s Website, where they were posted yesterday.

A version of this news article first appeared in the Inside School Research blog.

Events

Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Data Webinar
Using Integrated Analytics To Uncover Student Needs
Overwhelmed by data? Learn how an integrated approach to data analytics can help.

Content provided by Instructure
Professional Development Online Summit What's Next for Professional Development: An Overview for Principals
Join fellow educators and administrators in this discussion on professional development for principals and administrators.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education 'Widespread' Racial Harassment Found at Utah School District
The federal probe found hundreds of documented uses of the N-word and other racial epithets, and harsher discipline for students of color.
1 min read
A CNG, compressed natural gas, school bus is shown at the Utah State Capitol, Monday, March 4, 2013, in Salt Lake City. After a winter with back-to back episodes of severe pollution in northern Utah, lawmakers and Utah Gov. Gary Herbert will discuss clean air legislation and call for government and businesses to convert to clean fuel vehicles.
Federal civil rights investigators found widespread racial harassment of Black and Asian American students in the Davis school district north of Salt Lake City, Utah.
Rick Bowmer/AP Photo
Education Tiny Wrists in Cuffs: How Police Use Force Against Children
An investigation finds children as young as 6 and a disproportionate amount of Black children have been handled forcibly by police officers.
15 min read
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Nam Y. Huh/AP
Education Gunman in 2018 Parkland School Massacre Pleads Guilty
A jury will decide whether Nikolas Cruz will be executed for one of the nation’s deadliest school shootings.
3 min read
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Amy Beth Bennett/South Florida Sun Sentinel via AP
Education Briefly Stated: October 20, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read