Improvement networks can help districts come up with new approaches to educational problems—but more isn’t necessarily better. A new study finds school improvement networks often fall short when it comes to the rigor needed to make sure solutions in one school can apply elsewhere.
“In [continuous improvement] cycles, there’s this idea that you plan, you do, you study and then you act on that. But a lot of the networks drop off the study and act, they just plan and they do, and then they make decisions in the same way they always made decisions ... not based on the evidence that they collected,” said Elizabeth Chu, the executive director of the Center for Public Research and Leadership at Columbia University, which conducted the study.
Continuous school improvement is a cyclical process intended to help groups of people in a system set goals, identify ways to improve, and evaluate change. The approach has gained significant traction since the passage of the Every Student Succeeds Act, whose resulting state plans to implement the law incorporated continuous improvement models, including networks of schools that work together to test solutions to common problems. Over two years, CPRL researchers tracked the progress of nine school improvement networks supported by the Bill and Melinda Gates Foundation. The foundation supports 32 school networks run by 24 intermediary groups to develop ways to increase high school graduation and college and career readiness for Black and Latino students and those from low-income backgrounds.
Most networks decided early on to allow individual schools to come up with and test their own solutions to a common problem of practice.
“They wanted really localized-context-specific solutions,” said Ayeola Kinlaw, study co-author. “School by school, you would see different ideas being tested, but you could not compare apples to apples across the system. The data that was being collected didn’t allow the network as a whole, and certainly not the hub, to evaluate where there were common successes across networks.”
Rather, the two networks that did successfully scale up effective interventions allowed schools to test interventions from a collection gathered together based on scientific evidence, with the goal of figuring out which interventions would work best for different children in local contexts and why. The effective networks also made sure schools tested interventions in common ways that allowed them to compare results across schools and decide which practices should be piloted in new areas.
Network leaders in most cases found it “aspirational” to have all schools collaborating with each other directly, but the researchers found school leaders worked better when paired or matched with small groups testing solutions to a particular problem, which were then scaled up through a central hub. But the researchers found the most effective networks were also those with the most diverse combinations of schools, both geographically, rural and urban, and serving populations of different kinds of students.
“Presumably [as a district leader] you’re joining a network because you want to make decisions and accelerate improvement in ways that you hadn’t before,” Chu said. “And so looking at how the teams are functioning gives you a window into whether people actually are making decisions in different ways. How rigorous is [intervention] testing across individual teams? And to what extent does the process allow teams to learn from each other in a coordinated and meaningful way so that ... you can make smart decisions because you have access to a shared knowledge base that you otherwise would not have access to.”
The researchers recommended school leaders in networks:
- Focus on equity;
- Develop rigorous routines to evaluate the effectiveness of interventions;
- Facilitate meaningful collaboration among groups in the network;
- Engage district staff; and
- Reflect regularly to identify and address areas for improvement.
