Carrying out schoolwide reforms may be even harder than anticipated, a study scheduled for release this week suggests.
The report by the RAND Corp. looks at the first two years of efforts to “scale up” comprehensive reforms in a sample of schools participating in the New American Schools initiative. NAS is a nonprofit corporation that was created by the private sector in July 1991 to finance whole-school designs for improving student performance. Since 1995, it has concentrated its work on 10 states and districts that have agreed to implement such designs in at least 30 percent of their schools by 2000.
Based on case studies of 40 schools in seven of the districts, RAND found that about half were “implementing” or “fulfilling” a design--meaning that its core elements were evident across the school. But after two years, close to 45 percent were below that level, with four schools still only in the planning stages.
“We’re basically, in our analysis, providing a cautionary tale about how difficult it is to grow reform quickly,” said the study’s primary author, Susan J. Bodilly, a senior social scientist at the Santa Monica, Calif.-based think tank.
“We want to have a ‘buyer beware’ sign out there,” she added. “Don’t think that you can just buy this off-the-shelf technology, plug it into a school, and then things are going to improve. Much of it depends on the schools and the districts themselves.”
The researchers did not look at the effect of the various designs on student achievement. RAND began tracking such outcome data in the spring of 1996 and expects to begin reporting those results within the next year.
The findings come at a particularly crucial time for such whole-school programs. Schools and other entities are now gearing up to spend $150 million in new federal dollars, including $120 million in Title I money, for the adoption of comprehensive school reforms designed to raise student achievement.
Expanded Mission
It’s also a decisive time for New American Schools. The Arlington, Va.-based organization is seeking $5 million a year from foundations to greatly expand its mission--from a group that works with a small number of design teams to an umbrella organization that would market large numbers of designs to districts throughout the United States. The new organization would also push for whole-school design more generally, and work with districts on creating a more supportive environment for such reforms.
“There is a larger role that needs to be played to help comprehensive school reform succeed,” John L. Anderson, the president of NAS, said in an interview last week.
He described the results of the current study, based on only 40 of some 700 NAS schools, as encouraging. “You’ve got half that are intensively moving ahead and the others that are along a continuum,” he said, “and I think you would expect that.”
A Rushed Match
The RAND study found that a complex array of factors affected whether designs were actually carried out in schools. Schools differed greatly in their ability to implement reforms, design teams varied widely in their capacity to help schools, and districts offered varying levels of support. Elementary schools also had much higher levels of implementation than did high schools.
In general, the process used to match schools with designs was rushed, according to the researchers. NAS selected the participating jurisdictions in March 1995 and asked them to begin putting designs in place in at least 10 percent of their schools by that September.
Schools where educators felt that they adopted a design without fully understanding it or that they were forced to adopt a design showed lower levels of implementation than schools that were well-informed and had freedom of choice. Schools that experienced a turnover in leadership or that reported internal strife among the faculty also had lower levels of implementation. The selection process has since been modified to help schools better grasp the NAS-sponsored designs.
Assistance Varied
But the study also found that how well the designs were implemented depended, in part, on the capacity of the design teams.
Designs such as Roots and Wings and Expeditionary Learning that focused on the core elements of schooling--curriculum, instruction, student grouping, assessments, and professional development--tended to have higher levels of implementation than more ambitious designs that also addressed issues such as school governance or community engagement.
Designs also were more likely to be enacted if the design teams had stable leadership and a core group of people able to assist schools; clearly communicated what the design entailed; effectively worked with districts to secure resources, such as professional-development days; and supported the schools with whole-school training, materials, on-site facilitators, extensive training days, and checks on the quality of implementation.
Three designs--Authentic Teaching, Learning, and Assessment, or atlas, the Modern Red Schoolhouse, and the National Alliance for Restructuring Education--showed lower levels of implementation than did others. But the researchers cautioned that the scores might be affected by the particular sample of schools they chose and the districts in which they were located.
Political Turmoil
The study found that districts varied greatly in their ability to focus on and support the work of the design teams. The highest levels of implementation were in Memphis, Tenn., San Antonio, Kentucky, and Cincinnati. The lowest levels were in Philadelphia, Miami-Dade County, and Pittsburgh.
Districts tended to have higher levels of implementation if they had stable leadership that strongly supported the designs, were free of political crises, had a culture of trust between schools and the central office, provided some school-level autonomy in such matters as budgets and hiring, and provided more resources for professional development and planning.
Rand found that, on average, it cost $162,000 per school to implement a design in 1996-97, including both new money and the reallocation of existing dollars. The majority of funds--76 percent--were spent on personnel, such as paying the salary of an on-site facilitator or paying teachers for their training time.
The average San Antonio school committed 56 percent more personnel, 27 percent more planning time, and 14 percent more training days per design than the average school in all other districts.
In some districts, political turmoil overshadowed the attention paid to the designs. Districts such as Cincinnati and Philadelphia faced labor-management conflicts and budget crises. Pittsburgh had an intensive redistricting debate and a superintendent nearing retirement. Miami-Dade had to cope with a dramatically growing number of students, many of whom lacked strong English skills. In some sites, it was also unclear how the work of the design teams fit in with the larger reform agenda.
‘Messy, Organic Model’
Far from being an orderly process, the researchers found, "[o]ur analysis revealed a messy, organic model of change.”
That model could become even messier as states decide how to allocate the $120 million in federal Title I money set aside for research-based, whole-school designs such as NAS under the Obey-Porter Comprehensive School Reform Initiative, enacted by Congress last year. Another $25 million will go to schools through another Department of Education account, and $5 million will be spent on evaluating the project.
“I think the idea of supporting comprehensive school reform is the way to go,” said Ms. Bodilly, the RAND study’s primary author. But, she added, “so much is going to depend on how the states choose to manage this effort, and whether they choose to add burdensome processes or to support good practice.”
The NAS proposal for supporting such changes calls for the creation of an umbrella organization for “comprehensive, design-based assistance to schools and districts.”
New American Schools would use the lessons it has learned to create standards for comprehensive, design-based assistance organizations. It also would help districts restructure their policies and practices to support such efforts, build national awareness of the need for comprehensive reform, increase the capacity of existing NAS design teams, and expand the supply of new teams. In addition, the organization would recruit and train a field staff to work in participating states and districts.
