What makes one low-performing school turn around and build momentum over time, while another, seemingly similar school tries the same strategies but continues to struggle?
It’s not just particular programs or practices, but the interplay of school implementation with district policies and support, according to the Institute of Education Sciences’ Turning Around Low-Performing Schools project—the most comprehensive federal research on such schools to date.
“There’s not a lot out there on how you know that a school has turned around—and will stay turned around rather than just jumping up for a year,” said Rebecca Herman, a managing research analyst and school improvement expert for the American Institutes of Research. The AIR collaborated with Policy Studies Associates, the Urban Institute, and Decision Information Resources on the project.
While final reports will not be issued until later this year, researchers released the results of their four interconnected longitudinal studies of 750 chronically low-performing schools, in Florida, North Carolina, and Texas, during a symposium at the Society for Research in Educational Effectiveness conference here Sept. 6.
Comparing Apples
The project used an independent method to identify chronically low-performing schools and track them from 2002-03 through 2007-08; researchers developed their own identification system because other methods to identify persistently low-performing schools for the School Improvement Fund or No Child Left Behind Act accountability differed from state to state and did not include student growth. The study schools included the lowest 5 percent of schools in each of the three states, with achievement in the bottom 15th percentile for that state and less than 40 percent student growth over time in both reading and mathematics.
Student achievement was based on the performance in each school’s highest grade—typically 5th for elementary schools and 8th for middle schools—and student growth was calculated on the basis of the change in students’ performance between grades 3 and 5 for primary schools and 6 and 8 for middle schools.
“What one state may consider a low performer, another may not,” said Michael Hansen, a member of the project team and a longitudinal-data research associate at the AIR, based in Washington. “We wanted to compare apples to apples ... and we wanted to find authentic improvement in performance, rather than statistical-measurement error or demographic changes.”
About half the schools identified as initially low-performing were able to show some signs of improvement within three years; another 35 percent showed no increase in student-achievement status or growth.
“Study Weighs Pros, Cons of Teacher Turnover.”
But 15 percent of schools were considered true turnarounds: They improved the number of students reaching proficiency in math or reading by at least 5 percentile points, with student growth rates in the 65th percentile statewide.
The study looked back at performance during the last three years of the study to ensure early improvement continued over time.
Turnaround rates varied considerably by state and subject, with schools much more likely to improve poor performance in mathematics than in reading, and only 3 percent to 4 percent able to improve in both subjects at once. Those two-subject-turnaround schools were more likely than other schools to report low turnover of highly qualified teachers and more technical assistance with data use.
Need to Read
Still, the lower reading-improvement rates caused “serious misgivings” for Jennifer O’Day, an AIR principal research scientist not involved in the project.
“Let’s face it, for kids to graduate from high school and do well in life, they really need to read, and read well, and if we aren’t helping them do that, we have a real problem,” Ms. O’Day said. “This is particularly true as we move toward the new Common Core State Standards,” which have an increasing focus on literacy across the curriculum.
In another of the project’s studies, Brenda J. Turnbull, a principal associate, and researcher Erikson Arcaira, both with the Washington-based Policy Studies Associates, conducted site visits and in-depth interviews at 36 of the schools identified as initially low-performing in 18 districts across the three states. Case-study schools did not know they had been identified as improving or not, and neither did those assigned to visit the schools.
Data use, targeted student interventions, and teacher collaboration topped the most common strategies at the schools deemed to be turnaround schools, while more schools that did not improve used new curriculum or professional development. A related study in the project found strategies such as extended content periods and schedule changes were more likely to be used at improving middle schools than primary schools.
Though strategies overlapped in both types of schools, Ms. Turnbull argued they were only surface similarities.
“There were some schools where there was a data wall, the numbers were in everybody’s face all the time, …but when we asked them what was central to your improvement effort, there was variation,” and teachers did not have a clear sense of how the data were being used, she said.
Working in Combination
Improving schools tended to combine strong leadership and data use with strategic teacher recruitment, management, and “intensive” professional development. Ms. Turnbull noted that most of the schools, regardless of whether or not they improved, said they used professional development, but researchers only counted that training as “intensive” if it was ongoing throughout the school year and designed to address the specific issues raised in the school’s turnaround plan. For example, turnaround schools were more likely to provide professional development specifically on how to analyze and use student data to improve instruction.
“It’s different from just saying ‘professional development was our central thing'—there’s so much drive-by professional development,” she said.
The differences between improving and non-improving schools became clearer when researchers looked at groups of strategies and supports. For example, more than a third of the turnaround schools implemented a combination of data use and targeted interventions, compared with fewer than one in 10 of the schools that didn’t improve, Mr. Arcaira said.
In fact, the researchers found turnaround schools implemented on average fewer improvement strategies, 2.3 during the period studied, than schools that did not improve, which used 2.6. Staff in 17 percent of the schools that did not improve reported “too many strategies” were being used.
“We found multiple interlocking strategies that formed a framework for improvement, rather than discrete strategies that one might call a silver bullet. The sense of coherence, collaboration, leadership, and support came across as different atmospherically in the way people described them,” Ms. Turnbull said. “Certainly, just throwing multiple reform efforts at the school is not a good way to go.”
Schools that made substantial improvement were more likely to have accountability pressure and support from their district leaders, and to have heard explicitly how their school’s improvements fit into district reform. “It really confirmed the importance of these local factors at implementation and the complex network of policies, programs, practices, and supports in school turnaround,” Mr. Arcaira said.
