Memphis district leaders have announced that, after six years and $12 million, they are pulling the plug on their closely watched experiment to put schoolwide improvement models in every public school in the city.
“When you look at the expenditure and the results we were getting, it did not seem to me to be worth it,” said Johnnie B. Watson, who took over as the superintendent of schools last October.
His decision, announced at a school board meeting last month, is expected to reverberate far into the national “whole school” reform movement. Under Mr. Watson’s predecessor, Gerry House, the 118,000-student district became Exhibit A in the push for comprehensive school reform in the mid-1990s.
Until then, few, if any, school districts had attempted to institute so many comprehensive reform programs on such a large scale. For her efforts, Ms. House was named the National Superintendent of the Year in 1999 by the American Association of School Administrators.
|
“It’s a very bad thing for the whole comprehensive-school-reform movement to have the flagship district abandon the idea,” said Robert E. Slavin, a co- founder of Roots & Wings, the improvement model that 40 of the Memphis district’s schools were using.
New American Schools, the national, nonprofit corporation that has sunk $500,000 of its own money into the Memphis program, also criticized the change of course.
“We know there’s no silver-bullet, no one-size-fits-all strategy that’s going to work everywhere,” said Karen Hinton, who is the vice president for external affairs for the Arlington, Va.-based group. “But for a new superintendent to come in and sort of wipe out everything is very troubling.”
Pressure To Perform
But the reviews from inside the district last week were more positive, at least in some quarters. Some of the biggest cheers came from the local teachers’ union, whose members have complained the improvement models were costing them too much time and paperwork.
“For a long time now, teachers of Memphis city schools have been trying to relay a message that these models just weren’t working,” said Lynette E. Tabor, the president of the 5,600-member Memphis Education Association. “All teachers want to do is be allowed to teach.”
Superintendent Watson said he had based his decision in part on a six-month study of the restructuring models that was carried out by district researchers. He requested the study after becoming interim superintendent in the spring of last year and setting out to visit all 165 schools in the district, which is Tennessee’s largest.
“Everywhere I went, teachers and parents would tell me the reform models were not working,” he said. “Admittedly, I would go into some schools and the three teachers who would meet with me with the principal there would applaud the reform model. The same three teachers would then sneak around to me as I was getting into my car and tell me to get rid of the model.”
The recently completed report points to stagnant or declining test scores for city students on state tests in mathematics, reading, and English. Under Tennessee’s school accountability system, the district, which has long ranked at the bottom among districts statewide, is facing increasing pressure to raise its scores. Twenty-six of the 48 low-performing schools identified by state education officials as possible targets for state takeover are in Memphis.
Only three of the 18 largely off-the-shelf programs used in the district—Widening Horizons Through Literacy, a homegrown approach, and two national models, Core Knowledge and Voices of Love and Freedom—showed appreciable gains, according to the report. Those models are used in a total of only 11 Memphis schools.
Roots & Wings, the most popular of the improvement designs, was characterized as showing “little positive impact.” One model, the ATLAS Communities design, even had a negative impact on test scores, according to the district’s analysis.
In an accompanying survey of 4,500 of the district’s 7,000 teachers, a frequent complaint was that students needed more basic-skills instruction than many of the models provided.
“I just put the objective evidence together with the subjective, anecdotal evidence and made a professional decision,” Mr. Watson said.
Despite his newcomer status as superintendent, Mr. Watson is a veteran of the Memphis school system. He was deputy superintendent under Willie W. Herenton, who served as the superintendent from 1979 to 1991 and is now the city’s mayor. Mr. Watson accepted an offer from the school board to retire early when Ms. House became superintendent in 1992. After Ms. House left the district in April of last year, he returned with strong backing from the mayor.
Success or Failure?
The disappointing picture evoked by the in-house study differs from that of some other reports on Memphis’ pioneering school improvement efforts.
In a report presented at a national education conference in April, Steven M. Ross, a University of Memphis researcher, concluded that students in the city’s restructured schools were on the whole racking up bigger learning gains than students in schools that had not yet put improvement models in place or were just beginning to do so. He also found that the advances typically came about after two years, and that models that seemed to work in one school sometimes did not work in another.
The gains that Mr. Ross documented from 1995 to 1999, however, began to drop off in 2000, the year Ms. House left, Mr. Ross said in an interview last week.
“Things began to deteriorate even in the fall of 1999, and we think people were relaxing a bit in terms of zeal,” he added.
Florence Calaway, the principal investigator on the school system’s study, said the reports differed because the researchers used different methodologies.
Mr. Ross, for example, used “value added” achievement data. Proponents of that method, which is relatively new, claim it is fairer and more accurate because it measures students’ progress against their own past performance, thus avoiding the kinds of problems caused by comparing students from different socioeconomic groups, for instance.
Ms. Calaway and her co- investigators looked at students’ norm-referenced scores and percentile rankings, which measure how students perform in relation to one another.
For their study, the school system researchers tracked test scores of individual students who had been taught under one of the improvement models for at least a year. The study compares scores from each of the three years before the new models were put in place in their schools with those from up to five years afterward. To look at the impact of individual models, the researchers compared the test-score gains for schools using each model against the across-the-board academic progress for all city schools.
One potential problem with the trend data in the study, some study critics noted last week, was that it does not track a single group of students over time. For example, while almost all the schools in the system had been working with a reform model for two years, only 50 schools in the study sample had been at it for more than five years, and many of those schools are among the poorest and historically the lowest- performing schools in the district, a factor that would tend to depress scores.
But the district researchers said that socioeconomic differences were not a major issue, because three-quarters of Memphis students are poor enough to qualify for federal free- or reduced-price school lunches.
Meanwhile, even proponents of some of the reform models used in Memphis said last week that it was hard to judge from the data which studies were right.
“The truth probably lies somewhere in the middle of both extremes,” said Henry M. Levin, the founder of Accelerated Schools, a school improvement design that was used in 13 Memphis schools this past school year.
Next Steps
Part of the problem in assessing the new findings was that the full text of the report did not become available, even to the school board, until 11 days after Mr. Watson made his announcement.
Both of the evaluations of Memphis’ experiment agree on one point: Schools that adhered more closely to the program models tended to have better student-achievement outcomes than those that did not. Such findings led proponents of comprehensive approaches to school improvement to wonder why programs that seemed to work could not stay in place.
“I thought the program worked, if it’s done as is,” said Arthur M. Hull, the principal of A.P. Hill Elementary School in Memphis, which has been using Roots & Wings since 1995. He led a group of 20 Roots & Wings principals in the city who tried unsuccessfully to meet with the superintendent earlier this year to ask him to let them keep their own models.
Mr. Watson said he had refused to meet with anyone associated with the school improvement designs while the study was under way in order to keep the report—and his decision—from becoming “tainted.”
Mr. Hull, who was set to retire last week, has accepted a job with Success for All, the Baltimore-based improvement program that gave birth to Roots & Wings.
The principal and his fellow petitioners were represented in the first wave of schools that volunteered to take on a comprehensive reform design from the start. Schools that were slower to come on board with the schoolwide strategies were told they had to adopt such a model within three years.
“One of the lessons from Memphis is that you can’t impose something, because it creates a backlash,” Mr. Ross said.
Ms. House, who left the district to head the Institute for Student Achievement, a public-private partnership in Long Island, N.Y., said she had to mandate participation in order to “bring about change for every school and for every student.”
“Even if you eliminate the models you’ve not eliminated the changes the models were intended to bring about,” she said. “They were intended to change the attitudes, the behaviors, the assumptions about students, and the culture of the people in the school system, and that does not go away.”
In place of the strategy that Ms. House promoted, Superintendent Watson introduced an eight- point plan for improving Memphis schools. It calls for instituting new curriculum frameworks already being developed by the district, beefing up efforts at teaching reading, and buying new reading textbooks.
Mr. Watson also wants to provide extra staff members to help schools better monitor, mentor, and coach their teachers. Teachers could, though, continue to use the “best practices” they learned from the restructuring programs—even though the district will no longer provide financial or programmatic support for those models.
“We need something that all of our schools can implement to improve reading and math scores,” said Barbara U. Prescott, Memphis’ school board president, who backed Mr. Watson’s decision. “You have to figure out when you abandon a reform strategy, and you don’t abandon it when it’s working. But there has been very little gain.”
