I’m knee deep in old NCLB documents, and ran across the Department of Education’s NCLB song. NCLB represented not only a major shift in federal education policy, but an embrace of policy/PR boosterism that’s enough to make all of us giggle (Remember Armstrong Williams?). Back from 2002, here are the NCLB lyrics:
We're here to thank our president, For signing this great bill, That's right! Yeah, Research shows we know the way, It's time we showed the will!
No matter how catchy the ditty, a song can’t carry a fundamentally flawed law. That’s where Tom Toch and Doug Harris come in. They’ve penned a thoughtful commentary in this week’s Ed Week about the future of NCLB (Salvaging Accountability). It’s an important one, because it recognizes that NCLB conflates the school’s contribution to student learning with what students bring to the school to begin with. Essentially the argument is that:
1) “It’s critical in any accountability system that the metrics used to judge performance reflect accurately the contributions of those being judged.”
2) “As a measure of school performance, however, [the NCLB] snapshot strategy is flawed. Because student populations vary greatly from school to school, and because family income, parental education, and a host of other non-school-related factors have a major influence on students’ learning, some schools have to improve student achievement a lot more than others to get their students up to state standards. The federal law is unforgiving of such schools. As a result, it gives an unfair advantage to schools with students from privileged backgrounds, and it fails to measure what matters most: how much students learn during the school year.”
3) The Department of Education’s Growth Model Pilot offers little improvement over the current rating system because it relies on a projection model - i.e. are students on target to be proficient in a 3 year window? - rather than a true growth model.
4) The new NCLB should dump the projection model, and focus its sanctions on schools that are both low in terms of their growth, and low in terms of their proficiency. And there’s no reason to wait for reauthorization - this could all happen via regulations.
No commentary can do it all, so here are some issues to ponder for their next round. The goal of Toch and Harris’ proposed system is to make measurement of school performance a more fair and effective enterprise. Why not take the leap and dump 100% proficiency altogether? That way, we could narrowly tailor our sanctions to schools that are low-performing compared to the schools we already have.
And if we’re going to go full throttle on value-added models, we can’t just punt the measurement problems. For example, Toch and Harris write, “value-added calculations have larger margins of error than NCLB’s proficiency ratings, but because they measure what’s most important in judging schools—student learning gains—their statistical shortcomings are more than worth tolerating.”
A poorly designed growth model is no better than the poorly designed proficiency model that we have now, and no one knows this better than New Yorkers. Value-added systems that have literally no relationship between two years’ value-added measures are still bad public policy. In short, beware the silver bullet.