Data used in most state accountability systems does not provide useful information to improve instruction, and policymakers developing new assessments and accountability systems should plan more for data use on the front end, according to an analysis by Boston College researchers.
That’s hardly a new criticism, but the National Education Policy Center At the University of Colorado in Boulder, which sponsored the study, is hoping to get legislators thinking about new approaches with model legislative language based on the report.
“There’s too much being written and not much being done,” said Henry I. Braun, an education professor at the college and co-author of the analysis. “These reports are like stones in a lake; you get a big plop and then it fizzles out. We thought it was a great idea to say here’s what legislative language might look like.”
Braun and Boston College education chair Andrew Hargreaves, sponsored by the center with backing from the Ford Foundation and the Great Lakes Center for Education Research and Practice, looked at the use of performance data across several sectors, from education to sports and business.They found many indicators used for performance accountability—standardized test scores in classrooms or the number of steps a player takes on a soccer field, for example—are the easiest to measure but also the easiest to game. Moreover, they argued, these types of data often provide too little detail and arrive too late for them to be useful in improving instruction or administration.
“Through the ‘90s if you go back to the Improving America’s Schools Act, it encouraged states to get moving [on accountability],” Braun said, but “by the time [No Child Left Behind] came about, Washington looked around and said states haven’t done much. In part the harshness of NCLB was a reaction to the somewhat lackadaisical approach that came before.”
The report called for policymakers developing accountability systems to plan in advance what they value and create a “balanced scorecard” of how it can be assessed, rather than simply using indicators that can be easily measured. “Cultures of high pressure and high threat to achieve constant short-term gains often lead to negative consequences,” they argued. “Like the ill-fated Everest expedition team of 1996, organizations can too easily become gripped by ‘summit fever’ as they try to reach their targets, with the result that they lose sight of their core purpose and self-destruct in the process.”
Among the report’s recommendations:
• Ensure data is accurate and of high quality;
• Evaluate both support and performance and tie performance assessments back to help, be it tutoring for students or professional development for teachers;
• Move from targets that focus on thresholds to those focused on growth, so that indicators focus on improvements.
“We need to think about an accountability system that serves continuous improvement ... and to do that you need to have a coherent system of assessment,” Braun said. “In some ways it’s a Sisyphean task, but you can make that hill a little less steep and the boulder a little less heavy if you think about these things in advance rather than trying to fix them after the fact.”
Based on a dozen proposals in the report, Kathleen Gebhardt, an attorney and the executive director of Children’s Voices, a non-profit law firm that deals with children’s issues, developed model legislative language for potential changes to state or federal accountability systems. (This part of the project was paid for entirely by the National Education Association-backed Great Lakes Center, according to Jamie Horwitz, NEPC spokesman, as it would be too close to a lobbying activity for the Ford Foundation.)
It’s interesting for NEPC, which runs a separate think tank review project, to be pulling a page from think tanks on how to get education research before policymakers. Horwitz told me this will be more common going forward: “Increasingly what NEPC is trying to do is not just underwriting research on a area of education or problem of education, but also a legislative solution. We want to provide concrete ways to make a fix.”
What do you think?
Want more research news? Follow @SarahDSparks on Twitter for the latest studies, and join the conversation.