UPDATED

Do the new “A through F” and similar accountability systems states designed under the Obama administration’s No Child Left Behind Act waivers do a good job of recognizing how schools are doing when it comes to educating poor and minority students?

Not so much, according to a new report released Thursday by the Education Trust, an organization in Washington that advocates for such students.

Ed Trust took a look at the A-F-type systems in three states that are sometimes viewed as leaders in education redesign—Florida, Kentucky, and Minnesota—and found that in all three cases, school ratings are “not a powerfull signal of the performance of every individual group of kids.”

For instance:

• In Florida, which rates schools on an A-F scale, the average proficiency rate for African-American students in “A” schools is lower than for white students who attend “C” schools.

• Similarly, Kentucky puts schools in one of three baskets: “distinguished,” “proficient,” and “needs improvement.” And in the Bluegrass State, African-American students in schools earning “distinguished” ratings have a lower math proficiency rate than white students in schools that are designated as “needs improvement.”

• And in Minnesota, schools deemed “celebration eligible” or “reward” schools did about as well when it comes to math results for African-Americans students as schools singled out for improvement did on results for white students.

What’s more, Ed Trust found that many of the schools that were given a gold star from their states aren’t necessarily closing the achievement gap. In Florida, for instance, 39 percent of ‘A’ schools that have data for African-American students had lower proficiency rates for that group in 2014 than in 2013. And 45 percent of schools that earned a ‘B’ with data for Latinos saw their performance slip in reading during the same period. Results in Kentucky for black students at high-ranking schools were pretty similar.

(It’s important to note that Ed Trust selected those states in part because they had good, useable data available.)

Importantly, the study doesn’t answer a key question regarding the waivers: Are traditionally overlooked groups of students, such as English-language learners, students in special education, and racial minorities, showing greater improvement under the new systems than they were under NCLB Classic?

It’s just too early to say, Ed Trust concludes.

But the organization is worried that if schools where students in particular subgroups are underperforming are given good ratings by their states, they won’t have much incentive to fix the problem. Plus, there won’t be much transparency for parents, who often select schools based in part on their ratings.

So what should the U.S. Department of Education do? Ed Trust thinks the agency needs to put the focus back on equity when it issues waiver-renewal guidance, which will likely happen later this fall.

This isn’t the first time that Ed Trust has sounded the alarm on subgroup performance and the waivers. The organization was one of the first to question the use of “supersubgroups,” which allow schools to combine different groups, such as English-learners and students in special education, for accountability purposes.

In responding to the report, Raymonde Charles, an Education Department spokeswoman, said, “We appreciate Ed Trust’s commitment to supporting historically disadvantaged students. The Department of Education will continue to work with states to support their efforts to improve their accountability systems for schools and achievement for all students.”

And Carissa Miller, the deputy executive director of the Council of Chief School Officers, said that states are “committed to supporting all students and ensuring every child meets the goals of college and career readiness. We know the simplistic pass/fail determinations under AYP did not work. Today, states are taking more thoughtful approaches to figuring out how to put systems in place that help them better identify areas of need and target support. Still, regardless of what accountability system a state uses, states are and continue to be committed to focusing on closing the achievement gap and looking for ways to improve.”