Amid a nationwide backlash against testing, states were expected to jump at the chance to design accountability systems that judge schools on measures other than test scores alone—from specific offerings such as Advanced Placement courses to systemic factors such as school climate.
But while 42 states plus the District of Columbia have these waivers under the No Child Left Behind Act, only 18 took advantage of the opportunity to use multiple measures that went beyond the NCLB-era factors of test scores, participation in the assessments, and graduation rates in high school.
The vast majority of states’ new accountability systems just slice and dice test scores to rate their schools, according to research and analysis by a team of researchers, led by University of Southern California assistant professor Morgan Polikoff, and provided to Education Week. And of the 18 states that use other measures, many simply rely on results from different tests, such as the SAT and ACT college-admission exams, to help judge schools.
“For all the griping about testing, the data show that a lot [of states] still say math and reading scores are all that matter,” said Mr. Polikoff, whose waiver analysis provided the basis for a critique of state accountability systems published in the January issue of Educational Researcher. “On the other hand, collecting new measures can be expensive, can be time consuming. There’s also a lack of creativity in these state offices.”
But there are exceptions, researchers found.
Kentucky will incorporate principal- and teacher-evaluation results into school ratings starting in the 2015-16 school year. New Mexico includes the results of a classroom survey taken by students. And Oregon factors in students who are “on track” in 6th grade, measured by rates of chronic absenteeism.
“We need to make sure we are attending to those things that are most important,” said Rob Saxton, the deputy superintendent of public instruction for the Oregon Department of Education, speaking at a panel on multiple measures at a meeting last month of the Council of Chief State School Officers in Washington.
Despite being offered flexibility from the U.S. Department of Education, fewer than half of the states with No Child Left Behind Act waivers elected to go beyond traditional test scores, participation rates, and graduation rates in their school accountability systems. Among those that did, here’s a list of other measures being used.
SOURCE: Researchers Morgan S. Polikoff, Andrew J. McEachin, Stephani L. Wrabel, and Matthew Duque; Education Week
But states vary widely on what they see as important, based on their decisions in crafting accountability systems as part of the federal waiver process.
Attendance, especially in elementary school, is a popular nontest indicator among those waiver states looking to use alternative measures in weighing schools. So is advanced coursetaking, especially participation in AP classes or the International Baccalaureate program.
Three states—Kentucky, South Dakota, and Alabama—factor the rate of highly effective, or ineffective, teachers in each school into a school’s rating (based on the states’ new teacher-evaluation systems), according to the USC-led research. Nevada seems to be the only state to use remediation rates in the state’s higher education system as part of its high school accountability system.
And across the spectrum, states using multiple measures rarely attached more than 30 percent of a school’s grade to nontest factors, the data show.
Much Needed Leeway
The NCLB waivers—first rolled out in 2011 by the U.S. Department of Education—provided much needed flexibility from the constraints of the outdated law, the current version of the Elementary and Secondary Education Act. In classrooms across the country, there was growing concern about narrowed curricula that focused too much on math and reading, the core subjects the NCLB law addressed.
“I think under No Child Left Behind, there was far too much focus on a single test score and on the proficiency cut score, those bubble of students around that narrow bar,” U.S. Secretary of Education Arne Duncan said at a U.S. Senate education committee hearing on waivers last year. “What we wanted to put in place were multiple measures and the focus on growth and gain, so there weren’t incentives to teach a narrow band of students, but to help every child.”
While all of the federally approved accountability systems under waivers incorporate student growth, and not just absolute proficiency rates, Mr. Duncan’s hope that more states would incorporate multiple measures hasn’t been fully realized.
Even in states that incorporate multiple measures, the state chiefs say the priority—as it should be—is on capturing student achievement and growth. And test scores are the easiest way to do that.
In 2012, Oklahoma tried to elevate other factors, such as a school climate survey and measures of parent and community engagement, and made them worth 33 percent of a school’s grade. But district and school leaders balked when the first report cards came out in 2012, criticizing the new system for penalizing them due to factors they couldn’t control.
For the 2013-14 school year and beyond, these so-called “whole school” factors are merely bonus points. Among those, however, is a fairly unusual indicator: the graduation rate of low-performing 8th graders.
Kerri White, an assistant superintendent for educator effectiveness, said Oklahoma’s data system shows that before entering high school, students struggling the most in 8th grade are most likely to drop out. The state wanted to offer bonus points in its accountability system to high schools for successfully graduating that group of students.
“We really wanted to give them an incentive to focus on those kids,” Ms. White said. “We’re trying to find what the leading indicators are for academic achievement. Test scores are a lagging indicator.”
Self Audits in Kentucky
Kentucky is one of two states that incorporates program reviews into school grades. In Kentucky, this is a self-audit schools do of their programs and curriculum, worth 20 percent of a school’s grade. It asks schools questions such as: How well does the principal engage parents? Does the school offer high-quality arts programs? Do teachers have access to relevant professional development?
Schools must provide evidence to the state for their answers, which are then subject to a state education department audit as a check-and-balance.
“We wanted to make sure our schools and districts offered a balanced curriculum,” said Kentucky Education Commissioner Terry Holliday. “We had schools eliminating art and music and P.E.”
In addition to program reviews, the state grades high schools on their students’ ACT scores. And once the state’s new teacher-evaluation system is fully operational, the percentage of effective (or not effective) teachers in each school will count in the rating system.
Mr. Holliday said state officials studied whether to include dual enrollment and success in Advanced Placement courses as indicators of college readiness, but found it didn’t alter the school ratings. Down the road, however, Kentucky is eyeing an additional measure: the success of high school graduates in college.
“It’s not just all on the colleges to make sure students succeed,” Mr. Holliday said. “We think our high schools ought to carry some responsibility.”
Even in states that embrace multiple measures, such as New Mexico, state leaders say test scores must remain the most important part of a school accountability system.
“Our best way of measuring students’ achievement is through assessment,” said New Mexico’s education chief, Hanna Skandera. “But you have to be thoughtful. There is a role and a place for assessments.”
In New Mexico’s A-F grading system, its multiple measures are accounted for in up to five bonus points for schools, out of a grading scale of 100. Schools get bonus points for the results of student surveys, attendance, reduction in habitual truancy rates, and college- and career-readiness indicators in high school. And although the state may tweak its grading system here and there, Ms. Skandera said she doesn’t foresee any big changes.
“We want a consistent bar,” she said. “The richness of what we are able to capture is resoundingly better than anything we’ve ever had before.”
