In the decade since the No Child Left Behind Act was launched, research has documented shortcomings and unintended negative consequences of the law’s adequate yearly progress, or AYP, provisions.
Today, we have more than 10 years of lessons learned and policy recommendations for improving school accountability systems, but we wonder whether state and federal officials will make the most of them.
Two of the most basic lessons of the NCLB era are that proficiency rates are a poor measure of school effectiveness (if they can be called a measure at all), and a narrow focus on English/language arts and math leads to a narrowing of curriculum and instruction.
In 2011, the U.S. Department of Education offered states the opportunity to put this knowledge into action. In exchange for developing new state-specific accountability plans that measured school performance and identified low-performing schools, the department offered states waivers from many NCLB mandates—most notably those on AYP. And, as of June, 42 states and the District of Columbia had won approval for their flexibility waivers.
While states had ample opportunity to make dramatic changes in accountability policy, our analysis... found that they did not, on average, do so."
But while states had ample opportunity to make dramatic changes in accountability policy, our analysis (recently published in Educational Researcher) found that they did not, on average, do so.
In short, we examined the rules for identifying “priority” and “focus” schools according to the waiver process and found a mixed bag of improvements over, and in some cases duplications of, many of the problems of NCLB accountability.
Frustratingly, some of the most glaring issues arise in areas where states had plenty of room for choice. One of these concerns the subjects to be tested and used for accountability. Under the waiver systems, most states (28 of the 42 we analyzed) will continue to use test results from only English/language arts and math for classifying priority and focus schools based on their students’ performance. Even though testing in science is required in all states, few states made the common-sense decision to use these test results as part of their school identification process.
A second problem is states’ decisions to focus overwhelmingly on proficiency rates to measure school effectiveness. As is well known by now, this measure severely penalizes schools that serve more diverse student bodies, a factor largely outside a school’s control. And while some states developed growth measures that come closer to capturing the impact different schools have on children’s learning, many waivers do not use them to identify low-performing schools.
Federal officials also created issues of their own in developing their waiver rules.
First, states are required to identify a fixed percentage of schools (15 percent) as low-performing each year. This arbitrary cutoff dramatically reduces the number of schools identified, as highlighted in a recent report from the New America Foundation, but sends the confusing message that a school at the 16th percentile is meaningfully better than one at the 14th.
Second, the Education Department prohibited states from accounting for student characteristics in measuring student growth (despite allowing states to set different proficiency-rate targets for different demographic groups). Thus, those states that chose to incorporate growth in identifying low-performing priority and focus schools—which is a good thing—are forced to use performance measures that are likely biased by student characteristics—a bad thing.
Many of the problems of NCLB’s accountability system were discovered shortly into, if not prior to, implementation; yet, nothing was done to address them. The waivers were written as short-term policies that would provide a faster fix to any issues or unintended consequences that arose.
The Education Department recently announced that states with approved waivers must apply for an extension while awaiting congressional reauthorization of the Elementary and Secondary Education Act (the law currently known as No Child Left Behind). While it could require fixes to address some of the problems of the new waiver systems, the department appears to have bowed to concerns about perceived federal overreach by reducing the rigor of this review process.
The extension process is an ideal opportunity for states to make adjustments to their accountability systems in the short term and address larger problems or weak spots such as the ones highlighted earlier (and in more detail in our full paper). Unfortunately, if the state waivers are not fixed, the problems we have identified will continue to plague accountability practices in the near future—leading to the wrong schools being identified for intervention. Given this, states should take advantage of the opportunity offered by the waiver renewal to make changes.
To point the way toward improved systems for identifying low-performing schools, states might look to some exemplars that we believe have designed the strongest systems. Massachusetts and Michigan, in particular, stand out for increasing the validity, reliability, and fairness of their priority and focus classifications. Both states use non-test-based measures (e.g., attendance, educator effectiveness, school climate) for identifying the lowest-performing schools, expanding the desired outcomes of schooling beyond test scores alone. Both states also rely on test results in multiple subjects, such as science and/or social studies, reducing the incentive to narrow the curricular focus in schools.
Massachusetts considers multiple years of data to calculate achievement growth, which increases the measure’s reliability, and draws on a combination of growth and proficiency rates, enhancing the classifications’ validity. And Michigan has focused on measuring achievement gaps as within-school differences between the highest- and lowest-performing students, irrespective of demographic characteristics. As such, the state is reducing the penalty for enrolling a diverse student body, while simultaneously increasing the focus on all students within a school, not just those near the proficiency target.
These two states are not the only examples of creative systems that improve on AYP. But they do provide us with the perspective that states can be responsive to what we know about the design of effective accountability systems, moving beyond the minimum requirements of the waivers to construct thoughtful systems that are more likely to target the schools that most need support and intervention.
As it stands, we think that the nation has taken a few steps forward on accountability, but that we still have miles to go when it comes to reducing the unintended consequences of standards-based accountability. Let us hope that it does not take another decade before states and the Education Department respond to the research and improve school accountability.
