Federal

In Whole-School Reform, Staying True to Model Matters

By Debra Viadero — May 14, 2007 7 min read

In 1997, Congress established a hefty state grant program to encourage poor schools to invest in proven, off-the-shelf models for improving learning. By 2006, though, the Comprehensive School Reform Demonstration program—or CSRD, as it’s commonly known—had effectively, and quietly, died. Though still on the U.S. Department of Education books, the program made no new grants to states in that year or since.

Now, along comes a study offering a fitting, if somewhat ironic, epitaph for the program: Models matter.

“It wasn’t simply the fact of adopting the practices that seemed to make a difference on students’ achievement,” said Kerstin Carlson LeFloch, a study co-author. “It’s the whole bundle of practices.”

The federal study, which was conducted by researchers at the American Institutes for Research, or AIR, a nonprofit research group based in Washington, tracked progress in 650 elementary and middle schools, half of which were implementing one of eight different, packaged school improvement models, such as Success for All or Modern Red Schoolhouse, and half of which were using no formal schoolwide program to boost student achievement, between 2001 and 2004.

View the papers on comprehensive-school reform, posted by the Journal of Education for Students Placed at Risk.

Funded in 2000 when President Clinton was still in office, the $7.5 million study was completed last year. But the Bush administration, which does not count comprehensive school reform among its priorities for K-12 education, did not publicize it. The findings were published last fall in a report by the research group and again last month in a series of articles in the Journal of Education for Students Placed at Risk.

What the results show, somewhat surprisingly, is that schools not using the reform models did not look, in practice, all that different from the comprehensive- reform schools. Three-quarters of schools in both groups used practices that were very close to those that model developers saw as ideal for their programs. In other words, if the treatment school in the same district was using a model that emphasized applying test data to make instructional decisions or involving parents more in their children’s schooling, the comparison school tended to have the same sorts of practices in place.

“I think what schools perceive as good ideas in comprehensive school reform spread quickly to other schools,” said Daniel K. Aladjem, the study’s lead author and a principal research scientist at the AIR.

Time to Ripen

Possibly because the two groups of schools looked so much alike, the researchers found no differences between them, overall, in the gains students made on mathematics and reading tests over the five years of the study. Upon closer examination, though, the pattern became more complex: Between the third and the fifth year of implementation, schools that had stayed true to their school improvement models experienced achievement gains at a rate outpacing those of the comparison schools. The boost in achievement was all the more notable because the experimental schools had started out lagging behind the control schools. In the experimental schools that adhered less closely to the program guidelines, on the other hand, test-score gains were no different from those in other schools.

“In short,” one of the journal papers on the subject concludes, “CSR ‘works’ when external models are implemented faithfully and consistently for three to five years.”

The learning gains also varied, depending on the program that schools used. Success for All, a program developed by researchers from Johns Hopkins University in Baltimore, appeared to produce the largest academic improvements of any of the models studied, the study found, and to have schools that adhered closely to its program.

Some other models, though, stood out for their progress in building “social capital” in schools—fostering, in other words, an atmosphere in which teachers expressed a feeling of collegiality, collaborated in lesson planning, took part in extensive professional-development activities, and shared a sense of mission for their schools.

According to the report, two models produced the most gains in that regard: Authentic Teaching Learning and Assessment for All Students, or ATLAS, and the Accelerated Schools Project. Because some of the practices that go hand in hand with the development of social capital, such as professional development, were linked to long-term achievement gains, Mr. Aladjem speculated that gains in those schools might catch up, over time, to those in the Success for All schools.

“By definition, if you’re going to build a professional-development community, it’s going to take longer,” he said.

NCLB Catalyst

On Examination

In a five-year federal study, researchers from the American Institutes for Research tracked the life cycle of comprehensive-school-reform programs in hundreds of elementary and middle schools across the country. Featured programs were:

Accelerated Schools Project
Developed in 1986 by Henry M. Levin, then an education and economics professor at Stanford University, the program is known best for its focus on providing accelerated instruction to all children, including those labeled “at risk.” The national program is now based in Storrs, Conn., and has satellite training centers throughout the country.

ATLAS Communities
Short for Authentic Teaching Learning and Assessment for All Students, ATLAS Communities is the 1992 brainchild of four prominent school improvement thinkers: Yale University psychologist James P. Comer; Howard Gardner, a Harvard University psychology professor; Theodore R. Sizer, now a Brown University professor emeritus; and Janet Whitla, a former president and chief executive officer of the Education Development Center, a Newton, Mass.-based research group. The national office is based in Cambridge, Mass.

Co-nect
Established in 1992, Co-nect is a data-driven schoolwide-improvement program. The model is now overseen by Pearson Achievement Solutions, an arm of Pearson Education, a for-profit educational media company based in Saddle River, N.J.

Expeditionary Learning/Outward Bound
Though the Outward Bound outdoor-adventure program has been around for 60 years, it did not evolve into a formal whole-school-improvement program until 1992. The program, based in Garrison, N.Y., makes real-world instruction and community-service projects a key feature of school curricula.

Modern Red Schoolhouse
This K-12 design was founded in 1992 by the Hudson Institute, a think tank based in Washington. The program is now housed at the Modern Red Schoolhouse Institute in Nashville, Tenn.

Success for All
Perhaps the nation’s best-known school improvement program, the Baltimore-based Success for All was founded in 1987 by husband-and-wife researchers Robert E. Slavin and Nancy A. Madden, both of Johns Hopkins University in Baltimore.

Turning Points
Based at the Center for Collaborative Learning, in Boston, Turning Points is a middle school reform model developed in 1999. It grew out of a landmark national report of the same name that was published by the Carnegie Council on Adolescent Development.

Urban Learning Centers
This model for the pre-K-12 grades was a collaboration of the Los Angeles Unified School District and United Teachers of Los Angeles.

SOURCE: Education Week

The study’s findings are based on longitudinal surveys of principals, teachers, and district administrators in the participating schools, successive waves of test-score data, and case studies conducted at 24 of the experimental schools. Because the study was not a strict scientific experiment, the authors note, the findings are not definitive. But they mirror, to some degree, those of others suggesting that schoolwide changes take three to five years to bear fruit and that fidelity to the program is key to success.

“When teachers are completing surveys under confidential circumstances, and they’re being asked about behaviors and not their perceptions, that data is pretty valid and reliable,” said Laura M. Desimone, an assistant professor of public policy and education at Vanderbilt University in Nashville, Tenn., who critiqued the study. “The real strength of this set of studies is that they were able to look at the full process, from adoption to effects, to sustainability,” she said.

Two-thirds of the experimental schools, the researchers found, sustained a relationship with the developers of their program models for more than three years; some endured more than a decade. One-third of the schools had ended their formal program relationships by the end of the 2003-04 school year.

Yet, even though they had dropped their programs, those schools carried on business as usual, albeit to a lesser degree. They continued many of the same practices they had picked up from the brand-name improvement programs they dropped, according to the study.

Another surprise in the study: The advent of the federal No Child Left Behind Act, which was signed into law in January 2002, prodded a few of the schools in the study sample to pick up the pace at which they were implementing their program models.

Among the smaller sample of case-study schools, Ms. LeFloch found that when schools were identified to be in need of improvement under the law, they speeded up implementation of their reform programs rather than choosing to discard them.

“I think it was the case that they already had a strategy to turn to,” said Ms. LeFloch, a principal research analyst at the AIR, “and I think that’s in keeping with NCLB’s theory of action.”

Forthcoming Studies

Scenarios for Reform

Schools in the comprehensive school- reform study sustained their efforts to varying degrees.

Nonreform never takes hold and ends quickly.

Momentary reform flourishes briefly but quickly dies or is overtaken by another reform.

Nominal reform establishes itself but in name only and is eventually abandoned.

Resident reform establishes itself but persists in name only.

Transient reform establishes itself, changes the system, and then passes away, leaving little evidence that it ever occurred.

Temporary reform establishes itself, changes the system, but gradually gives way to the forces of inertia and persists in name only.

Sustained implementation overtakes whatever preceded it so completely that it is institutionalized as the status quo and ceases to be “reform.”

Sustained implementation without a single sustained reform occurs when schools maintain their model, yet continue to adjust it, after the formal relationship with the developer has ended.

SOURCE: American Institutes for Research

The AIR study is among several federally financed research projects on comprehensive school reform coming out after the program effectively ended.

Researchers at WestEd, a San Francisco research group, have been tracking 1,000 schools across the country since 2002 as part of an evaluation of the $1.6 billion federal program. The findings from that $8 million study, however, have been in federal review for more than a year, according to Naida Tuschnet, the program director for evaluation research at WestEd.

Researchers at the University of Michigan in Ann Arbor have also been conducting a long-term study of various schoolwide models.

All those findings will fill out a research track record for comprehensive school changes that has been fairly spotty until now. While some programs boast strong evidence of success, most have yet to muster definitive proof they can turn schools around. The movement also suffered from widely publicized failures in some districts, such as Memphis, Tenn., that had been implementing the schoolwide programs on a wide scale.

In the meantime, many reformers have shifted their sights to the district, rather than schools, as the locus for change, sometimes and sometimes not incorporating comprehensive-improvement models into their reform mix. And the No Child Left Behind law, with its emphasis on discrete parts of the school curriculum, particularly reading and math, operates in some respects at cross-purposes with some of the most popular change models for schools.

Still, Mr. Aladjem said, “I do think there has been an indelible impact from the program, … and some of that would be lost in the demise of CSR.”

Coverage of education research is supported in part by a grant from the Spencer Foundation.
A version of this article appeared in the May 16, 2007 edition of Education Week as In Whole-School Reform, Staying True to Model Matters

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning
Classroom Technology Webinar Making Big Technology Decisions: Advice for District Leaders, Principals, and Teachers
Educators at all levels make decisions that can have a huge impact on students. That’s especially true when it comes to the use of technology, which was activated like never before to help students learn
Professional Development Webinar Expand Digital Learning by Expanding Teacher Training
This discussion will examine how things have changed and offer guidance on smart, cost-effective ways to expand digital learning efforts and train teachers to maximize the use of new technologies for learning.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Biden Calls on Schools to Host COVID-19 Vaccination Clinics for Kids 12 and Up
The president is focusing on vaccinating children ages 12 and older as concerns grow about the Delta variant and its impact on schools.
2 min read
President Joe Biden speaks in the South Court Auditorium on the White House campus, Wednesday, June 2, 2021, in Washington.
President Joe Biden speaks in the South Court Auditorium on the White House campus on June 2.
Evan Vucci/AP
Federal How Political Backlash to Critical Race Theory Reached School Reopening Guidance
A lawmaker wants Miguel Cardona to repudiate the Abolitionist Teaching Network after federal COVID-19 documents referenced the group's work.
6 min read
Rep. Virginia Foxx, R-N.C., is seen at a press conference at the U.S. Capitol on March 9, 2021 in Washington.
Rep. Virginia Foxx, R-N.C., is seen at a press conference at the U.S. Capitol on March 9, 2021 in Washington.<br/>
Graeme Sloan/SIPA USA via AP
Federal Biden Team: Schools Can Go Beyond Trump Rules in Response to Alleged Sexual Misconduct
The Education Department's guidance, released July 20, states that Title IX rules from 2020 lay out "minimum steps" for educators.
3 min read
Symbols of gender.
iStock/Getty
Federal Fact Check: After Furor Over 1619 Project, Feds Adjust History and Civics Grant Plans
A previously obscure history and civics program has weathered a political storm, but what exactly has changed?
4 min read
Education secretary nominee Miguel Cardona speaks during a Senate Health, Education, Labor and Pensions Committee hearing on his nomination on Feb. 3, 2021, in Washington.
Education secretary nominee Miguel Cardona speaks during a Senate Health, Education, Labor and Pensions Committee hearing on his nomination on Feb. 3, 2021, in Washington.
Anna Moneymaker/The New York Times via AP