Seven years and $7 billion dollars later, the federal School Improvement Grants program seems in the final analysis not to have yielded much in terms of improved student achievement.
The Institute of Education Sciences on Thursday released its final evaluation of the program, which Congress killed last year.
“Overall, we didn’t find evidence that schools implementing SIG-funded models significantly changed student outcomes more or less than other similar low-performing schools,” said Lisa Dragoset, a senior policy researcher for Mathematica Policy Research, who co-wrote the evaluation.
She cautioned that the overall findings don’t “necessarily mean that no student benefited from attending a SIG school.” Maybe, but they’re still a far cry from the early results from the first two years of the program, in which about two-thirds of schools that began the improvement grants in the 2010-11 and 2011-12 school years showed boosts to their performance on state math and reading tests. By contrast, in the final evaluation, Mathematica and the American Institutes of Research found none of the program’s four school improvement models led to significant gains in students’ math or reading test scores, graduation rates, or college enrollment, when compared to similar struggling schools who had not participated in the grants. Researchers had compared student achievement at schools that barely did or did not make the cutoff to be identified for turnaround under the grant.
“There were such high expectations for the program when it first started, that these would become dramatically different and better schools,” said Robin Lake, the director of the Center on Reinventing Public Education at the University of Washington at Bothell. “They didn’t hit dramatically different; unfortunately, they didn’t even hit better.”
Tracking School Turnaround Practices
During the Bush administration, the SIG program funded state systems of support for improving schools. But with a massive infusion of cash from the federal economic stimulus in 2009, the Obama administration reinvented the grants as a means to push—and evaluate—four models to turn around chronically failing schools:
- Transformation, which required replacing the principal and adopting a value-added evaluation system for teachers and leaders that included student growth, as well as overhauling the schools’ governance, instructional practices, and learning time, and incorporating community school practices;
- Turnaround, which called for similar changes to the transformation model, but instead of implementing a new teacher evaluation system, it called to replace the principal and at least half of teachers outright;
- Restart, which involved reconstituting the school under a charter organization; and
- Closing the school and enrolling its students in higher-achieving schools in the district.
Evaluators found that by 2013, SIG-funded schools used about three more of the 35 turnaround practices studied—such as extended learning time and comprehensive instructional reform—than other, similar schools, but the difference wasn’t significant. Interestingly, nearly 9 in 10 of both the SIG-funded schools and other schools trying to improve made changes in instruction, like adopting a new curriculum. Fewer than half of SIG or other schools made “operational flexibility” changes, like giving principals more power over their budgets, hiring decisions, discipline, or school schedules.
That was in line with what Lake said she saw in an earlier study of turnaround practices at SIG schools in Washington. “Not much had really changed in the schools,” she said. “They were being asked to do different things, but the fundamental culture of the school, organization of the school, the fundamental design wasn’t reorienting toward dramatically higher intervention strategies, dramatically higher expectations, or dramatically better teacher training and support.”
Moreover, schools that used SIG money were more likely than other schools to increase student learning time or create community-centered schools, but they also stopped using those practices faster than other schools did after a couple of years, suggesting they may not have been able to sustain the initiatives as well after the grant money ran out. In spite of general similarities, the study found schools that used SIG money to implement the turnaround practices tended to be poorer and in more urban districts than schools that implemented the practices on their own.
Dragoset suggested the lackluster results of the program might have been the result of more schools both in and out of the grant program adopting similar turnaround practices, or that the individual turnaround practices were not as well implemented in some schools.
In an interview with Time this week, former Secretary of Education Arne Duncan argued the SIG schools’ progress has been “encouraging” and that grant-funded high schools “improved faster than all other public schools nationwide.” The school improvement evaluation did find that middle and high schools that adopted a turnaround model had students with higher math scores than schools that adopted a transformation model, but Dragoset said it was not clear whether that meant the overall turnaround model was more effective than the transformation model. And there were no similar differences among the models in reading or for math in elementary schools.
“As schools and states consider evidence-based approaches for turning around low-performing schools, it’s important for researchers to continue conducting rigorous studies to find out ‘what works’ to improve student achievement,” Dragoset said.
Lake agreed: “The amount of money spent here was a lot, so contextually we’re talking about billions of dollars spent to little avail,” she said. “That should give all of us pause when it comes to big new interventions. We need to have an evidence base before we ask everyone to implement them.”
A version of this news article first appeared in the Inside School Research blog.