Learning Their Lessons
"We get requests, letters, and telephone calls from all over the country: from individual schools, from school districts, from state groups. And what they're asking is Á 'Come and help us.' Well, we can't go and help them. Because, as I say to them, 'There are more of you than there are of us.' ''
--F. James Rutherford
Director of Project 2061 at the AmericanAssociation for the Advancement of Science
Some of the best-known school-reform networks are the victims of their own success. They started small but now encompass hundreds of schools. The Coalition of Essential Schools began with nine members in 1984. Today, it has 184 member schools and 585 more are considering membership. The Accelerated Schools Project has jumped from two pilot schools in 1986 to approximately 700. James P. Comer's School Development Program now comprises about 400 schools in 65 districts. All of these groups are struggling with how to get their ideas or practices out to a wider audience with integrity. "What we don't know yet,'' says Ann Lieberman, the co-director of the National Center for Restructuring Education, Schools, and Teaching at Teachers College, Columbia University, "is what you have to do to not lose the essence of these things when you scale up.''
To the reformers, the numbers they're dealing with are mind-boggling. But there are 80,000 public schools in the United States, and the number enrolled in one of the major reform efforts is a drop in the bucket. Moreover, research doesn't suggest much about how to spread innovations successfully. "The research in this area is so much a documentation of failure that it's hard to know what to make of it,'' laments Robert E. Slavin, the director of the Success for All network at Johns Hopkins University. "Really powerful demonstrations that scale-up of well-researched, well-worked-out methods can actually occur is one of the most important things that's needed right now.''
A 1990 report on replication by the Charles Stewart Mott Foundation found that successful replications share some common elements: evidence that the initial program is having the desired impact; careful planning; the ability to leverage community support, resources, and dollars; committed leadership; effective technical assistance; and sharing and communication.
Officials of some of the biggest reform networks recently reflected on what their experiences have taught them about "going to scale.''
A Clear Purpose
"Be clear about what you stand for. No one will rally to your flag if you do not have compelling ideas and represent a set of values with which site partners and nonsite partners can identify.''
--From "Getting to Scale: Lessons from the Experience of the National Alliance for Restructuring Education''
It seems obvious. The most successful reform networks are organized around powerful visions of teaching and learning. But how specific--or prescriptive--should that vision be? Bill Honig, the former superintendent of public instruction in California, argues that many school-improvement efforts are "too generic and too caught up in trivial issues.''
"Part of the problem is design,'' he asserts in the June issue of Phi Delta Kappan magazine: "Reforms are not typically organized around improving teachers' knowledge of content or enhancing their ability to collaborate to improve instruction. Á The lack of sophisticated curricular and instructional focus has plagued state and district restructuring initiatives, as well as such highly publicized national efforts as James Comer's School Development Program, Henry Levin's Accelerated Schools, and Theodore Sizer's Coalition of Essential Schools. Evaluations of these efforts show that only a few schools have undergone the radical curricular and instructional changes necessary to produce significant improvement in student performance.''
"We will not work with schools unless they've voted by at least 80 percent to take on the program. And we try to make sure that's an informed vote--that people know what they're getting into. It's taken very seriously. It's by secret ballot. Our experience is that's not very difficult to get, but the process of doing it is absolutely essential. Whenever we've shortchanged it in any way, we've always paid for it.''
--Robert E. Slavin
Director of the Success for All Network
When schools are reluctant partners, reform is fragmented and incomplete. So, many reformers place a premium on gaining the support and understanding of a school's members from the outset.
The Accelerated Schools Project expects sites to go through a months-long courtship that includes visits to other Accelerated Schools. It requires the written support of 90 percent of a school's faculty, as well as its student and parent representatives. "Informed consent is real important,'' says Henry Levin, the program's director, "and informed means really informed.''
"We do not bend it knowingly,'' he adds, "but we do know that some schools bend it in terms of what they tell us. We ask for signatures Á but there are people who will sign because they think the principal is pushing it.''
Many of the design teams funded by the New American Schools Development Corporation now require sites to sign formal agreements that spell out the mutual obligations on both sides. The Coalition of Essential Schools also has a number of conditions for membership. But it has reduced its demand for schools to secure a strong, initial vote of confidence.
"We used to require 75 percent of a faculty to vote in favor of joining the coalition,'' says Robert McCarthy, the Coalition's director of school development.
"We were told by many of our friends in the field that it creates unnecessary divisions in a faculty between the haves and the have-nots,'' he says. "So now, in our membership materials, what we say is a 'substantial majority.' ''
A study by researchers from Johns Hopkins University and Abt Associates, a Cambridge, Mass.-based research firm, suggests that many schools try to circumvent the buy-in process. Samuel Stringfield and his colleagues examined 25 schools that had implemented one of 10 reform strategies for educating disadvantaged children. In most cases, they found, schools adopted a reform strategy without carefully considering the alternatives or had it imposed from above by a principal or a school district. "When I started the study, I thought I'd find people who had a map and said, 'Here's where I am, and here's where I want to go,' '' says Stringfield. "The image that I wound up with is more like hitchhiking. People stick out their thumb and go. They so desperately want out of the town they're in that they just hitch onto the next ride.''
In at least one of the Coalition sites studied, the vote to enter the program passed only after the principal told the faculty that he would be in big trouble with the superintendent if the faculty voted no, and after the faculty explicitly agreed that while the school was joining, no teacher would be required to participate in any Coalition activity.
"We've been at this game for a decade . Á One hunch I have is that the schools that have really moved have been those where the faculty had real control over what was going to happen. Central Park East, University Heights, Thayer High School are, at their best, places which broke very sharply with the status quo and were able to sustain those changes and to adapt them as experience required because they were in charge.''
--Theodore R. Sizer
Chairman of the Coalition of Essential Schools
To undertake the ambitious changes in teaching and learning that many reformers envision, schools need substantial autonomy and authority: over curriculum, instruction, materials, hiring and firing, the organization and training of school staff, and the use of time and resources.
"We had a horrible superintendent a couple of years ago who said, 'You can be a Success for All school, but you also have to implement [our] curriculum.' So we had schools implementing reading two different ways,'' recalls Slavin. "We want to get as many guarantees as we can in advance that the school can pursue its vision of what it wants to become with the supportive noninterference of the district.''
According to a strategy paper prepared by the New American Schools Development Corporation, "Perhaps the key requirement, emphasized by all design teams, is the need for substantial school-level autonomy.'' This includes the freedom to demonstrate accountability in ways beyond that required by the state or district.
"The incentives for being bold are very, very weak,'' comments Sizer. "The most powerful incentive is a combination of real authority and highly visible accountability, where visible means, 'Come and look at our kids before you make any judgments.' This combination of authority and great public accountability is very attractive to the kind of confident and smart people that you want. And unless you have confident and smart people, you won't have any scale-up worth a tinker's damn.''
"If you just reform what's going on in the school and you don't reform what's going on in the district, in terms of management and organization, you're cooked. The experience of almost everybody who has looked at schools that work is that they work in spite of the system, not because of it. And they eventually fall apart because the system will get you every time.''
--Marc S. Tucker
President of the National Center on Education and the Economy
Many reform networks began with a focus on individual schools because they believe that change must occur school by school. This has led to some wonderful, but isolated, schools. "We have one school in Charleston, W.Va., that's absolutely fantastic,'' notes Slavin of Success for All. "They'll probably never have another one. But, if we have a choice now, our emphasis is on situations in which it's likely that one school will be the start, rather than the end, of the process.''
Almost every network has also lost schools to the vagaries of district politics: supportive principals who were reassigned; state and district regulations that ran counter to innovative practices; time and money for staff training that were held hostage in central offices.
So while reformers still understand that the real action is at the school site, many of them now require districts to make detailed commitments of support up front. And more and more of them are turning their attention to working with school districts. "We would never go into a district without the superintendent's or school board's endorsement,'' explains Edward T. Joyner, the acting director of the School Development Program. "But now, we're asking for a firmer commitment because, over the past 25 years, we've learned that you cannot do any meaningful school reform without a systemic orientation.''
Programs may ask districts to commit money for staff development and conference fees, pay for a facilitator in the central office, agree to the collection of data and periodic site visits, secure the support of individuals in key leadership positions, and provide guarantees about a school's autonomy.
The Accelerated Schools Project now hopes to develop whole school districts based on its model. It's working with the San Jose Unified School District in California to create a jurisdiction in which the central office and all of the schools will adhere to its principles. "We're still putting most of our effort into individual schools,'' says Levin. "But in the long term, we see the district as either a very supporting or limiting element.''
That recognition led the National Alliance for Restructuring Education to work with states, districts, and schools simultaneously. The partnership, which was formed by the National Center on Education and the Economy in 1990, now includes 56 schools in five states and four city school districts. "Schools can only be changed one by one,'' agrees Tucker, "but, in the long run, those changes cannot be sustained unless the larger system is also changed because what makes schools what they are lies mostly outside the school. The schools and that larger system must be changed in concert.''
A similar belief undergirds the plans of the New American Schools Development Corporation. The corporation will focus its replication efforts on five to seven jurisdictions that have pledged to provide a supportive "operating environment'' for schools. Among the features of such an environment are: publicly supported standards of achievement for virtually all students; a rich and reliable system of assessments; sources of assistance for schools in choosing and developing curriculum and instruction; and a management and governance system that provides schools with broad guidance, individual autonomy, and the support needed to achieve their mission.
"The single most important variable is leadership, 'strong, like-minded people who are willing to stay the course.' We have never succeeded in any measure where good leadership is lacking.''
--From "Getting to Scale: Lessons from the Experience of the National Alliance for Restructuring Education''
Everyone agrees that leadership is important, whether it's a charismatic and dedicated principal or a straight-shooting superintendent. But reformers have learned they need leadership from a number of quarters, not just a single individual. Otherwise, the superintendent leaves or the principal is fired and the reform falters. "Early in the history of the Alliance, we found ourselves betting on one horse, only to find that person moving on and the effort collapsing quickly,'' write Marc Tucker and Judy Codding in a paper that summarizes lessons learned by the National Alliance for Restructuring Education. "There must be a strong leader in a key position and that leader must be joined by others who are equally committed to much the same agenda and situated in key positions. This goes for states, districts, and schools.''
Frank Newman, the president of the Education Commission of the States, says at the state level, reformers have tended to focus on winning the support of governors and key policymakers. But they've paid less attention to state-level bureaucrats and individual legislators who can make or break reform. So when governors have stepped down, or been voted out of office, the reforms have been vulnerable. Newman now believes that reformers must retrace their steps and work with a much broader array of people than they had once thought.
Training and Support
"If you want to bring about fairly simple changes which are intuitive for people, then you can do it fairly briskly. But if the nature of the daily interactions between students and teachers, between students and technology, between teachers and teachers is going to be fundamentally different, people don't have experience in doing that. And if there's going to be more responsibility for placing education in the hands of local folks, then a McDonald's cookie-cutter kind of approach isn't going to work.''
Director of the Project Zero Development Group
The more complex the reforms, the harder they are to implement, and the more people need help. Virtually every reform group is struggling to provide ongoing, intensive support to educators in the midst of change. "The research on change says that if schools are going to implement various promising programs, of almost any type, they're going to need a long-term commitment at the school level and from the developer of the program,'' notes Stringfield, a professor of education at Johns Hopkins University. "That's the premise that makes scaling up so hard. It's clearly important that they get that Á but how in the world is it going to be provided?''
The response of most networks has been to train and support "coaches'' who work with schools on a regular basis. In 1991, the Accelerated Schools Project provided a five-day training program for teams of eight to 10 people from individual schools. A follow-up of the schools' progress a few months later revealed that one-third of the schools were floundering and another third were doing nothing. "We had made no provision for systematic follow-up,'' says Levin, the director of the program. "We were trying to cram into the heads of people what they were going to do for the next five to six years, and that's absurd.'' Since then, the project has developed a training-and-certification program for coaches who are based out in the school district. "The coaching model,'' Levin says, "has become the major vehicle for expansion.''
Reading Recovery relies on "teacher leaders,'' specially trained individuals who continue to work with children at the same time that they educate other teachers. The teacher leaders spend a year honing their skills at 23 regional training sites. Then they work with a class of 12 new teachers each year. But they always remain in touch with the previous year's group. "You do not stop and have everybody ready and do an intervention,'' says Gay Su Pinnell, the director of Reading Recovery. "People need to know, 'I will never learn all there is to know about this. I am part of Reading Recovery and, therefore, I am a continual learner.' ''
All of the nasdc design teams use coaches or facilitators, although the frequency of their interaction with schools and their availability varies widely. Success for All requires schools to designate a full-time facilitator at the school site. "We'll compromise on a lot of things,'' says Slavin, "but we've found we just cannot compromise on that. Somebody must be responsible for seeing through the change process at the school every day, down to the little details. Somebody needs to be worrying about the quality of the implementation all the time for something that is major.''
In their study of promising strategies for educating disadvantaged students, the Johns Hopkins researchers found access to technical assistance that is strategy-specific almost always had a positive effect. And ongoing professional development was consistently associated with higher levels of implementation.
But the problem for many networks is how to provide high-quality support as they expand in size. For almost all of them, the response has been to decentralize: from a small staff of experts at Yale or Brown or Stanford to a constellation of regional centers scattered across the country. Even so, studies suggest that the amount of guidance and assistance schools receive varies widely. And those sites receiving the least assistance typically show the least fidelity to the original concepts.
Many of the reform networks have also discovered that they need to provide more help with curriculum and pedagogy than they had anticipated. The School Development Program emphasizes the use of child-development theories and principles to improve student learning. But it recently hired someone to work with districts on curriculum and instructional issues. It has also sought alliances with other groups--like Project Zero--that can help teachers improve their curriculum planning, instructional delivery, and assessments. "In the past, we've not given a whole lot of guidance on curriculum,'' says Joyner, the acting director. "But what we're seeing now is that we need to do that.''
The National Alliance for Restructuring Education has commissioned a set of essays that illustrate best practices in curriculum and instruction, written by subject-matter experts and teachers. It's also developing a "national curriculum academy,'' where lead teachers can synthesize and distribute best practice. The Johns Hopkins study of 10 reform strategies found that, for all the programs, the aspect of schooling slowest to change was the content of the core curriculum.
A Sense of Connectedness
"One of the things that invariably runs down innovations over time is a feeling of isolation. The people within a school have got to feel that there's somebody elsewhere who cares a lot about what they're doing and speaks their language.''
--Robert E. Slavin
All of the reform groups have come to appreciate the importance of networks that allow conversation, support, and learning among groups of like-minded people. Many of them are trying to strengthen networks through newsletters, conferences, electronic linkages, and clusters of schools in close geographic proximity. "At the local level, there's been a movement that we've been promoting where schools are connected with each other and they create local support networks,'' Slavin says. The schools hold monthly meetings, visit each other, and send their more experienced members to work with new schools. Similarly, Project Zero has found that once teachers have the chance to visit schools where changes in teaching and learning are occurring, they can develop networks that take on lives of their own.
Anthony W. Jackson, the executive director of the middle-grades school-state policy initiative at the Carnegie Corporation of New York, says "providing intersite visitation and peer-to-peer reflection is really the primary vehicle that we believe will lead to scaling up. If you can have a few anchor--or mentor--schools, you can work off those and begin to develop relationships in a kind of exponential manner that allows people to learn firsthand from their peers how to do all this stuff.''
"Reading Recovery is very consistent. We've learned a repertoire of local adjustments, but we've also learned what's constant that we have to keep essential. If you're going to call it Reading Recovery, we're going to ask you to have certain characteristics in place.''
--Gay Su Pinnell
Director of Reading Recovery
The largest school-reform networks disagree sharply about what should be held constant and what should be allowed to vary school by school.
At one extreme are programs like Reading Recovery and Success for All, whose well-researched models depend on teachers following a precise approach to curriculum and pedagogy. At the other end of the spectrum are more philosophical approaches, like the Coalition of Essential Schools, that share common principles but encourage schools to devise their own solutions. In between are networks like the Accelerated Schools Project. It requires schools to follow a very specific process for analyzing problems and reaching consensus but does not mandate a particular approach to reading or mathematics.
"For philosophical approaches,'' argues Stringfield, "a major challenge is how to translate the philosophy into concrete instructional strategies.'' Studies have found that educators at the same school often interpret the Coalition's principles differently. And Stringfield argues that schools would benefit from "standard solutions'' to commonly recurring problems, such as how to organize class schedules to provide more time for learning.
But to some groups, providing such concrete models goes against their efforts to be intellectually respectful of educators. Many of the school designs are also works in progress. They rely on local sites as much to develop new knowledge as to replicate old techniques.
In contrast, says Slavin, "People like us and Reading Recovery believe that you have to have a program and something reasonably well worked out if it's going to be effective when it is implemented and to see that it can be implemented at all.''
Time and Costs
"We have learned from our experiences that you can't hit the schools with everything at once. But, on the other hand, you have to go very rapidly--a lot more rapidly than most theorists in this area would believe. Á We say that if teachers are not perceiving a marked difference in reading performance by December, then you're in big trouble. Teachers have been through so many false reforms that they need to see something in the fairly near term that convinces them life is now really different.''
--Robert E. Slavin
The big school-reform networks also disagree about how long it takes to restructure a school and how much it costs. Success for All expects schools to see concrete results within the first year. Henry Levin estimates that it takes six years to create an Accelerated School, although schools with two or three years' experience have shown "dramatic gains'' in student achievement, parental participation, student and teacher attendance, and better expectations and attitudes.
The Coalition of Essential Schools estimates that it can take 18 months to two years for a school to meet the requirements for membership, but there is no time line for full implementation of its principles. The New American Schools Development Corporation has kept its design teams on a breakneck pace that required them to develop their designs in one year and implement them fully in two.
Cost estimates are even murkier. Levin argues that the transformation to an Accelerated School can be made primarily by reallocating existing resources. The Coalition estimates that schools restructured according to its principles cost only 10 percent more than existing schools to operate.
"I'm convinced that one can take the typical high school and, if one successfully rearranges the resources, design a dandy school,'' says Sizer, the Coalition's chairman. He admits, however, that "the costs of the planning and retraining are considerable, the up-front R&D costs. And I personally underestimated that. I don't know what it is.'' Such costs, Sizer now says, should not be rolled into the operating expenses for schools "anymore than progressive business puts it in that way. Research and development is a line item on its own. Most public education fails to invest anything at all in this kind of work.''
"If you talk about Protestant churches, they all have the same book. But you can go to one church and see it looks like these folks are loving their neighbor and honoring God. You go to another and you say, 'Is this a church or a saloon?'''
--Edward T. Joyner
Acting Director of the School Development Program
Every reform group wants its ideas to spread with fidelity, which quickly leads them to the question of quality control. "There's a definite phenomenon in education reform--we call it 'grand faloonery' from [Kurt Vonnegut's novel] Cat's Cradle--in which schools want the name, they want the flag, but they actually don't want to do any reform,'' says Slavin of Success for All. "And we do everything we possibly can to look for that.''
Both Success for All and Reading Recovery gather detailed data on the progress of individual students and use them to help monitor whether schools are running into problems in implementing their model. The Accelerated Schools Project is trying to establish a data base to keep track of its schools. The School Development Program tries to track a school's test scores, its climate (as perceived by teachers, parents, and administrators), students' self-concept and attendance, and the rates of suspension and explusion. But Joyner says many schools and districts aren't used to collecting such information. The program also sends out people twice a year to assess whether school sites are faithfully replicating elements of the program.
The ultimate measure of quality is whether students perform at consistently higher levels. But most schools have not been engaged in the reform networks long enough to produce such data, except in anecdotal form. They decry the use of traditional multiple-choice tests to measure students' progress. But new forms of assessment that would be more appropriate are still under development.
As a result, it's often hard for schools to evaluate whether these models are worth pursuing. And it's equally hard for reformers to insure that schools are not joining their network in name only.
In general, it is rare that any reform network kicks out a school or a district. And some efforts lack a specific schedule or a set of benchmarks for evaluating progress. "Almost none of these programs notice when one of their schools doesn't do an important component,'' Stringfield of Johns Hopkins charges. "They have very few mechanisms in place to notice when lapses are happening. That would be a hell of a way to run a nuclear power plant.''
In the study of strategies for educating disadvantaged students, Stringfield found that none of the programs had left a "clear, uniform imprint on regular classroom instruction.'' And he was struck by how unevenly and incompletely programs were implemented at many sites.
So Stringfield began looking at the characteristics of organizations--like air-traffic controllers and electric-power companies--that operate with high levels of reliability. "If you assume that the program you have is valid, there is a literature out there that can tell you what things have to be in place for reliable implementation,'' he says.
High-reliability organizations, he found, share a number of characteristics: They consider failures a disaster and unacceptable; they have a clear sense of their primary mission; they have a set of understandable and replicable procedures; they recruit and train extensively because professional judgment is valued; and they have initiatives to identify flaws in their procedures and to propose and validate changes.
"There needs to be a quality-control component built into the system,'' he argues. "There can be diverse discussions about how to do that, but to just assume that these are good people and good people will go off and do good things is garbage.''
After two years, the National Alliance for Restructuring Education concluded that real change in what schools and districts did depended on the Alliance's being much clearer about what was expected and setting deadlines for accomplishing those things. As a result, it came up with 21 "indicators of core commitment''--things that someone should be able to see, feel, and hear in an Alliance school by the fall of 1995--as well as a set of 14 "vital signs'' to measure student success and system performance.
But many reformers are reluctant to become quality police, given the complexity of the changes they are asking schools to pursue and the different starting points of the schools. Such a role is also in conflict with the supportive, collegial atmosphere that many of the networks are trying to promote.
"If a school says it's an Essential School and then proceeds to deepen and intensify its ability grouping and its discipline-based teaching, to me that's a gross distortion,'' says Robert McCarthy, the Coalition's director of school development. "But if a school really stays on student as worker and doesn't recognize the need for schedule changes or anything that would disrupt the flow of life in the school, that's more a failure of nerve.''
"I think we have to be willing to tolerate these variations in order to have the ideas spread,'' he argues, "to have people talking about and interpreting the ideas, and to have at least these variations in practice begin to take place.''
Reasons for Hope
"We don't control schools. We don't have any money. We don't have any political power. All we can do is persuade people. And culture is not easy to change.''
Director of the Accelerated Schools Project
In the end, argues Richard Elmore, a professor at the Harvard graduate school of education, the school-improvement networks should be viewed as "national treasures. We should promote a culture of this kind of work. But they're very limited prototypes for a particular kind of social intervention, which is gathering up the faithful and converting them. And by definition, for a whole lot of reasons we can describe, they run out of gas, not the least of which is the extraordinary demands they make on the people who are in them.''
But others are more optimistic that the number and scope of such networks will continue to grow. And they reason that the lessons learned from them will enrich our information about educational change. "If you look at all these networks, for all the difficulties that you're talking about--which are more or less severe--there are more and more schools involved every year,'' argues Frank Newman of the Education Commission of the States. "There are more and more networks every year. So we're getting there.''