Federal

In Whole-School Reform, Staying True to Model Matters

By Debra Viadero — May 14, 2007 7 min read
  • Save to favorites
  • Print

In 1997, Congress established a hefty state grant program to encourage poor schools to invest in proven, off-the-shelf models for improving learning. By 2006, though, the Comprehensive School Reform Demonstration program—or CSRD, as it’s commonly known—had effectively, and quietly, died. Though still on the U.S. Department of Education books, the program made no new grants to states in that year or since.

Now, along comes a study offering a fitting, if somewhat ironic, epitaph for the program: Models matter.

“It wasn’t simply the fact of adopting the practices that seemed to make a difference on students’ achievement,” said Kerstin Carlson LeFloch, a study co-author. “It’s the whole bundle of practices.”

The federal study, which was conducted by researchers at the American Institutes for Research, or AIR, a nonprofit research group based in Washington, tracked progress in 650 elementary and middle schools, half of which were implementing one of eight different, packaged school improvement models, such as Success for All or Modern Red Schoolhouse, and half of which were using no formal schoolwide program to boost student achievement, between 2001 and 2004.

View the papers on comprehensive-school reform, posted by the Journal of Education for Students Placed at Risk.

Funded in 2000 when President Clinton was still in office, the $7.5 million study was completed last year. But the Bush administration, which does not count comprehensive school reform among its priorities for K-12 education, did not publicize it. The findings were published last fall in a report by the research group and again last month in a series of articles in the Journal of Education for Students Placed at Risk.

What the results show, somewhat surprisingly, is that schools not using the reform models did not look, in practice, all that different from the comprehensive- reform schools. Three-quarters of schools in both groups used practices that were very close to those that model developers saw as ideal for their programs. In other words, if the treatment school in the same district was using a model that emphasized applying test data to make instructional decisions or involving parents more in their children’s schooling, the comparison school tended to have the same sorts of practices in place.

“I think what schools perceive as good ideas in comprehensive school reform spread quickly to other schools,” said Daniel K. Aladjem, the study’s lead author and a principal research scientist at the AIR.

Time to Ripen

Possibly because the two groups of schools looked so much alike, the researchers found no differences between them, overall, in the gains students made on mathematics and reading tests over the five years of the study. Upon closer examination, though, the pattern became more complex: Between the third and the fifth year of implementation, schools that had stayed true to their school improvement models experienced achievement gains at a rate outpacing those of the comparison schools. The boost in achievement was all the more notable because the experimental schools had started out lagging behind the control schools. In the experimental schools that adhered less closely to the program guidelines, on the other hand, test-score gains were no different from those in other schools.

“In short,” one of the journal papers on the subject concludes, “CSR ‘works’ when external models are implemented faithfully and consistently for three to five years.”

The learning gains also varied, depending on the program that schools used. Success for All, a program developed by researchers from Johns Hopkins University in Baltimore, appeared to produce the largest academic improvements of any of the models studied, the study found, and to have schools that adhered closely to its program.

Some other models, though, stood out for their progress in building “social capital” in schools—fostering, in other words, an atmosphere in which teachers expressed a feeling of collegiality, collaborated in lesson planning, took part in extensive professional-development activities, and shared a sense of mission for their schools.

According to the report, two models produced the most gains in that regard: Authentic Teaching Learning and Assessment for All Students, or ATLAS, and the Accelerated Schools Project. Because some of the practices that go hand in hand with the development of social capital, such as professional development, were linked to long-term achievement gains, Mr. Aladjem speculated that gains in those schools might catch up, over time, to those in the Success for All schools.

“By definition, if you’re going to build a professional-development community, it’s going to take longer,” he said.

NCLB Catalyst

On Examination

In a five-year federal study, researchers from the American Institutes for Research tracked the life cycle of comprehensive-school-reform programs in hundreds of elementary and middle schools across the country. Featured programs were:

Accelerated Schools Project
Developed in 1986 by Henry M. Levin, then an education and economics professor at Stanford University, the program is known best for its focus on providing accelerated instruction to all children, including those labeled “at risk.” The national program is now based in Storrs, Conn., and has satellite training centers throughout the country.

ATLAS Communities
Short for Authentic Teaching Learning and Assessment for All Students, ATLAS Communities is the 1992 brainchild of four prominent school improvement thinkers: Yale University psychologist James P. Comer; Howard Gardner, a Harvard University psychology professor; Theodore R. Sizer, now a Brown University professor emeritus; and Janet Whitla, a former president and chief executive officer of the Education Development Center, a Newton, Mass.-based research group. The national office is based in Cambridge, Mass.

Co-nect
Established in 1992, Co-nect is a data-driven schoolwide-improvement program. The model is now overseen by Pearson Achievement Solutions, an arm of Pearson Education, a for-profit educational media company based in Saddle River, N.J.

Expeditionary Learning/Outward Bound
Though the Outward Bound outdoor-adventure program has been around for 60 years, it did not evolve into a formal whole-school-improvement program until 1992. The program, based in Garrison, N.Y., makes real-world instruction and community-service projects a key feature of school curricula.

Modern Red Schoolhouse
This K-12 design was founded in 1992 by the Hudson Institute, a think tank based in Washington. The program is now housed at the Modern Red Schoolhouse Institute in Nashville, Tenn.

Success for All
Perhaps the nation’s best-known school improvement program, the Baltimore-based Success for All was founded in 1987 by husband-and-wife researchers Robert E. Slavin and Nancy A. Madden, both of Johns Hopkins University in Baltimore.

Turning Points
Based at the Center for Collaborative Learning, in Boston, Turning Points is a middle school reform model developed in 1999. It grew out of a landmark national report of the same name that was published by the Carnegie Council on Adolescent Development.

Urban Learning Centers
This model for the pre-K-12 grades was a collaboration of the Los Angeles Unified School District and United Teachers of Los Angeles.

SOURCE: Education Week

The study’s findings are based on longitudinal surveys of principals, teachers, and district administrators in the participating schools, successive waves of test-score data, and case studies conducted at 24 of the experimental schools. Because the study was not a strict scientific experiment, the authors note, the findings are not definitive. But they mirror, to some degree, those of others suggesting that schoolwide changes take three to five years to bear fruit and that fidelity to the program is key to success.

“When teachers are completing surveys under confidential circumstances, and they’re being asked about behaviors and not their perceptions, that data is pretty valid and reliable,” said Laura M. Desimone, an assistant professor of public policy and education at Vanderbilt University in Nashville, Tenn., who critiqued the study. “The real strength of this set of studies is that they were able to look at the full process, from adoption to effects, to sustainability,” she said.

Two-thirds of the experimental schools, the researchers found, sustained a relationship with the developers of their program models for more than three years; some endured more than a decade. One-third of the schools had ended their formal program relationships by the end of the 2003-04 school year.

Yet, even though they had dropped their programs, those schools carried on business as usual, albeit to a lesser degree. They continued many of the same practices they had picked up from the brand-name improvement programs they dropped, according to the study.

Another surprise in the study: The advent of the federal No Child Left Behind Act, which was signed into law in January 2002, prodded a few of the schools in the study sample to pick up the pace at which they were implementing their program models.

Among the smaller sample of case-study schools, Ms. LeFloch found that when schools were identified to be in need of improvement under the law, they speeded up implementation of their reform programs rather than choosing to discard them.

“I think it was the case that they already had a strategy to turn to,” said Ms. LeFloch, a principal research analyst at the AIR, “and I think that’s in keeping with NCLB’s theory of action.”

Forthcoming Studies

Scenarios for Reform

Schools in the comprehensive school- reform study sustained their efforts to varying degrees.

Nonreform never takes hold and ends quickly.

Momentary reform flourishes briefly but quickly dies or is overtaken by another reform.

Nominal reform establishes itself but in name only and is eventually abandoned.

Resident reform establishes itself but persists in name only.

Transient reform establishes itself, changes the system, and then passes away, leaving little evidence that it ever occurred.

Temporary reform establishes itself, changes the system, but gradually gives way to the forces of inertia and persists in name only.

Sustained implementation overtakes whatever preceded it so completely that it is institutionalized as the status quo and ceases to be “reform.”

Sustained implementation without a single sustained reform occurs when schools maintain their model, yet continue to adjust it, after the formal relationship with the developer has ended.

SOURCE: American Institutes for Research

The AIR study is among several federally financed research projects on comprehensive school reform coming out after the program effectively ended.

Researchers at WestEd, a San Francisco research group, have been tracking 1,000 schools across the country since 2002 as part of an evaluation of the $1.6 billion federal program. The findings from that $8 million study, however, have been in federal review for more than a year, according to Naida Tuschnet, the program director for evaluation research at WestEd.

Researchers at the University of Michigan in Ann Arbor have also been conducting a long-term study of various schoolwide models.

All those findings will fill out a research track record for comprehensive school changes that has been fairly spotty until now. While some programs boast strong evidence of success, most have yet to muster definitive proof they can turn schools around. The movement also suffered from widely publicized failures in some districts, such as Memphis, Tenn., that had been implementing the schoolwide programs on a wide scale.

In the meantime, many reformers have shifted their sights to the district, rather than schools, as the locus for change, sometimes and sometimes not incorporating comprehensive-improvement models into their reform mix. And the No Child Left Behind law, with its emphasis on discrete parts of the school curriculum, particularly reading and math, operates in some respects at cross-purposes with some of the most popular change models for schools.

Still, Mr. Aladjem said, “I do think there has been an indelible impact from the program, … and some of that would be lost in the demise of CSR.”

Coverage of education research is supported in part by a grant from the Spencer Foundation.
A version of this article appeared in the May 16, 2007 edition of Education Week as In Whole-School Reform, Staying True to Model Matters

Events

English-Language Learners Webinar Helping English-Learners Through Improved Parent Outreach: Strategies That Work
Communicating with families is key to helping students thrive – and that’s become even more apparent during a pandemic that’s upended student well-being and forced constant logistical changes in schools. Educators should pay particular attention
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Top Federal Adviser on Puerto Rico's Schools Declares: 'We Have to Build Trust'
Chris Soto heads an Education Department team providing technical assistance and support for the U.S. territory's public schools.
4 min read
Martin G. Brumbaugh School kindergarten teacher Nydsy Santiago teaches her students under a gazebo at a municipal athletic park in Santa Isabel, Puerto Rico, on Feb. 4, 2021.
Martin G. Brumbaugh School kindergarten teacher Nydsy Santiago teaches her students under a gazebo at a municipal athletic park in Santa Isabel, Puerto Rico, on Feb. 4, 2021.
Carlos Giusti/AP
Federal Schools Could Count Nonbinary Students Under Biden Proposal
The Civil Rights Data Collection for this school year could also revive questions about inexperienced teachers and preschool discipline.
6 min read
Image of a form with male and female checkboxes.
iStock/Getty
Federal 'Parents' Bill of Rights' Underscores Furor Over Curriculum and Transparency in Schools
U.S. Sen. Josh Hawley's bill highlights how education issues like critical race theory will likely stay in the national political spotlight.
7 min read
Sen. Josh Hawley, R-Mo., speaks during a Senate Armed Services Committee hearing on the conclusion of military operations in Afghanistan and plans for future counterterrorism operations, Tuesday, Sept. 28, 2021, on Capitol Hill in Washington.
U.S. Sen. Josh Hawley, R-Mo., says "it's time to give control back to parents, not woke bureaucrats."
Patrick Semansky/AP
Federal Opinion It’s Not Just the NSBA That’s Out of Touch. There’s a Bigger Problem
Those who influence educational policy or practice would do well to care about what parents and the public actually want.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty