Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

School & District Management Opinion

The Value of Value-Added

By Rick Hess — April 21, 2010 4 min read
  • Save to favorites
  • Print

I’ve been meaning to do a longer postmortem on Florida’s Senate Bill 6. As I’ve noted before, I enthusiastically supported it even though I thought it a deeply flawed bill. The flaw? Its ham-fisted attempt to strip out one set of anachronistic strictures (governing tenure and step-and-lane pay scales) only to replace it with a set of test-driven processes that were almost equally troubling.

I’ll get to that eventually. But in travels to Boston and Houston yesterday, I had occasion to reflect on the often incautious faith in value-added assessment that underlies many efforts to rethink teacher evaluation and pay. The simple truth: value-added is an imperfect, imprecise, and fundamentally limited way to gauge teacher efficacy. That said, it’s still a helluva lot better than the status quo. If we’re honest and up-front about its limits and use it as tool that should be handled with care, I’m an advocate. The trick is to be openly and proactively critical about tools like value-added (or merit pay or school choice) so that we can design them thoughtfully and anticipate problems.

Unfortunately, too often--as in Senate Bill 6 or in proposals that districts should start to publicize the value-added scores of individual teachers--we are not using value-added carefully. Instead, it is frequently treated as a casual cure-all. Last night, at a terrific all-star Philanthropy Roundtable dinner panel (Andy Rotherham, Kati Haycock, Tim Daly, Alex Johnston, and Houston supe Terry Grier), I found myself getting nervous at how casually strong numbers on grades three to eight reading and math value-added were being treated as de facto determinations of “good” teaching.

There are all kinds of problems with this unqualified presumption. At the most technical level, there are a dozen or more recognized ways to specify value-added calculations. These various models can generate substantially different results, with a third of each result varying with the specifications used. When used for a teacher in a single classroom, we frequently only have 20 or 25 observations (if that). The problem is that the correlation of such results year after year is somewhere in the .25 to .35 range. Due to limited grade coverage, we can generate math and reading value-added for only about 25 to 30% of all teachers. As schools get smarter about rethinking staffing and integrating virtual instruction, it’ll be increasingly difficult to attribute a student’s performance to a single teacher (this is already a thorny question for districts where students receive substantial pull-out instruction or work with a reading coach).

A more fundamental question is how well reading and math assessments reflect the range of knowledge, skills, and habits of mind we want teachers to cultivate. I’m pretty sure they’re a good representation when it comes to serving academically at-risk students who historically haven’t been taught the basics; but I’m much less confident about how accurately they represent what we care about when it comes to students for whom foreign languages, music, or gifted instruction constitute a much larger share of the school day.

Now, we can measure teacher performance in various ways other than value-added scores. Such measures, along the lines that Tom Kane’s shop is piloting at the Gates Foundation, include familiar approaches like observation and more novel efforts like student feedback. Importantly, however, the validity of all of these techniques is being gauged by how tightly they correlate with grades three to eight reading and math scores. In other words, whether these are good measures of student learning is being gauged by how tightly they reflect value-added calculations. So, if our tests are flawed or are not capturing what we really care about, the proxy measures will be flawed or off-key in similar ways.

Again, I don’t think any of this is grounds for shying away from value-added. Using it is far, far better than the alternative. But it is, inevitably, flawed and imperfect. This means several things. One, we should be careful how we use it. Senate Bill 6 would have been much stronger had it knocked out tenure and step-and-lane pay but had given school districts more leeway in designing evaluation and compensation. Two, we should employ measures like using a couple years of data and trying to build complementary tests and assessments. Three, we need not to reflexively trust the assessments we have, but to sensibly and thoughtfully ask how reliable they really are.

And four, we need to do an infinitely better job educating parents and policymakers on the problems with current practices that value-added can address and then flag the pitfalls of such reforms and explain how they can be addressed. It’s going to be hard to win and maintain solid parental and legislative support for value-added systems--much less to win teacher backing--unless proponents can offer credible responses to concerns about inequities, inaccuracies, and distortions.

We have a tendency to fall in love with new solutions and then dismiss skepticism as opposition. That would be a serious mistake.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Attendance Awareness Month: The Research Behind Effective Interventions
More than a year has passed since American schools were abruptly closed to halt the spread of COVID-19. Many children have been out of regular school for most, or even all, of that time. Some
Content provided by AllHere
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Ensuring Continuity of Learning: How to Prepare for the Next Disruption
Across the country, K-12 schools and districts are, again, considering how to ensure effective continuity of learning in the face of emerging COVID variants, politicized debates, and more. Learn from Alexandria City Public Schools superintendent
Content provided by Class

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Letter to the Editor School Mask Mandates: Pandemic, ‘Panicdemic,’ or Personal?
"A pandemic is based on facts. A 'panicdemic' is based on fears. Today, we have both," writes a professor.
1 min read
School & District Management How 'Vaccine Discrimination' Laws Make It Harder for Schools to Limit COVID Spread
In Montana and Ohio, the unvaccinated are a protected class, making it tough to track and contain outbreaks, school leaders say.
4 min read
Principal and District Superintendent Bonnie Lower takes the temperature of a student at Willow Creek School as the school reopened, Thursday, May 7, 2020, in Willow Creek, Mont.
Bonnie Lower, a principal and district superintendent in Willow Creek, Mont., checks the temperature of a student as Willow Creek School reopened for in-person instruction in the spring.
Ryan Berry/Bozeman Daily Chronicle via AP
School & District Management Opinion 'Futures Thinking' Can Help Schools Plan for the Next Pandemic
Rethinking the use of time and place for teachers and students, taking risks, and having a sound family-engagement plan also would help.
17 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
School & District Management Opinion Best Ways for Schools to Prepare for the Next Pandemic
Being better connected to families and the community and diversifying the education workforce are some of the ways to be ready.
14 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty