School & District Management

RAND Finds Mixed Results For School Reform Models

By Debra Viadero — April 18, 2001 4 min read
  • Save to favorites
  • Print

A decade after New American Schools was founded to develop, test, and replicate comprehensive models for improving schools, the districts that are trying out those designs are still showing mixed results.

For More Information

Read the report, “Implementation and Performance in New American Schools: Three Years into Scale-Up,” from RAND‘s education research page. (Requires Adobe’s Acrobat Reader.)

That conclusion comes from the latest report from the RAND Corp. on the progress of the New American Schools initiative. Preliminary findings from the report, which is due to be released this summer, were presented here during last week’s annual meeting of the American Educational Research Association.

The aim of New American Schools, a private, nonprofit organization based in Arlington, Va., was to spearhead a national effort to foster school improvement models that resulted in dramatic improvements in student achievement. Toward that end, the organization in 1992 awarded start-up grants to 11 design teams with promising programs for improving schools.

New American Schools settled on seven of those designs when it began to move the programs out into 10 school districts across the country on a large scale in 1995.

The targeted designs included: Purpose-Centered Education, which was formally known as Audrey Cohen College; Authentic Teaching, Learning, and Assessment for All Students, or ATLAS; Co-Nect Schools; Expeditionary Learning Outward Bound; Modern Red Schoolhouse; America’s Choice Design Network; and Roots & Wings.

Researchers from RAND, a Santa Monica, Calif.-based think tank, have been tracking the initiative since 1991. But the new findings are based on studies begun five years ago as the school improvement models began expanding into school districts from Tennessee to Washington state.

Thanks to a federal program designed to encourage schools nationwide to adopt comprehensive reform models, the number of schools using one of the original New American Schools designs has since swelled to 3,000.

Gains Not Dramatic

The schools in the original 10 districts taking part in the New American Schools initiative were, for the most part, poorer than the national average and had higher-than-average concentrations of minority students, according to the RAND researchers.

Of the 163 schools they have been tracking over the past five years, 81—or about half—have made greater improvements in student achievement in mathematics than students in their overall districts. In reading, 76 schools, or about 46 percent, made similar gains.

“These do not meet the expectations of New American Schools for dramatic effects across the board,” said Susan Bodilly, the RAND researcher who presented the findings at the AERA conference.

Susan Bodilly

Her co-authors on a paper laying out those findings were Mark Berends and Sheila Nataraj Kirby, also of RAND.

Some school districts were more successful than others in showing student-achievement gains. Schools in Memphis, Tenn., and several districts in Kentucky, for example, saw the most improvement in mathematics. The biggest reading gains were chalked up in Cincinnati and in several districts in Washington state, according to the researchers.

Likewise, the designs themselves experienced uneven success. Of the eight Audrey Cohen schools examined, for example, five surpassed student- achievement gains made for their home districts in mathematics. But only two managed that feat in reading.

The most consistently successful reform model appeared to be Roots & Wings, an approach that piggybacks on Success For All, the systematic reading program created by researchers at Johns Hopkins University in Baltimore. Ten of the 21 schools using that design, which goes beyond Success For All to include other subjects besides reading, made progress in comparison with their districts in both reading and math.

Except for ATLAS and Expeditionary Learning Outward Bound, about half the schools that used the remaining designs outpaced gains for their districts in math.

Since the student-achievement measures differed from district to district, however, the researchers cautioned against reading too much into those results.

“Better and longer-term performance data are needed in order to make conclusive judgments about designs and their effects on school performance,” the researchers write in their paper.

Implementation Varies

Further muddying the performance picture, the districts varied greatly in the degree to which they were implementing the programs. Overall, the researchers found, two years into the start-up phase, about half of a subgroup of 40 schools that they looked at as case studies were implementing the designs at the target levels. And some models were being carried out more fully than others. Modern Red Schoolhouse, for example, tended to lag behind other designs in implementation.

But most of the variation the researchers found was occurring within schools, rather than from school to school.

“These attempts at whole- school reform did not affect the whole school,” the researchers write.

Influential Factors

The researchers identified factors that either hindered or helped schools and districts in their efforts to put in place the reform models. They found that:

  • The programs were adopted more fully in smaller schools and in elementary schools.
  • Teachers who cited their students’ lack of basic skills, parental support, or disciplined behavior as significant barriers to learning were less likely to fully embrace the models.
  • Having a stable team of consultants from the model programs who could communicate clearly, market their programs effectively, and conduct training for the entire school, rather than for a few teachers, bolstered the programs’ implementation.
  • The programs became more firmly ensconced in schools where they had the active support of principals and in systems that experienced consistent, district-level financial, regulatory, and political support for the models.
Related Tags:

A version of this article appeared in the April 18, 2001 edition of Education Week as RAND Finds Mixed Results For School Reform Models

Events

School & District Management Webinar How Pensions Work: Why It Matters for K-12 Education
Panelists explain the fundamentals of teacher pension finances — how they are paid for, what drives their costs, and their impact on K-12 education.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
Strategies for Incorporating SEL into Curriculum
Empower students to thrive. Learn how to integrate powerful social-emotional learning (SEL) strategies into the classroom.
Content provided by Be GLAD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Leadership in Education: Building Collaborative Teams and Driving Innovation
Learn strategies to build strong teams, foster innovation, & drive student success.
Content provided by Follett Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management The Harm of School Closures Can Last a Lifetime, New Research Shows
The short-term effects on students when their schools close have been well documented. New research examines the long-term impact.
5 min read
Desks and chairs are stacked in an empty classroom after the permanent closure of Queen of the Rosary Catholic Academy in Brooklyn borough of New York on Aug. 6, 2020.
Desks and chairs are stacked in an empty classroom after the permanent closure of Queen of the Rosary Catholic Academy in Brooklyn borough of New York on Aug. 6, 2020. A new study examines the long-term effects on students whose schools close.
Jessie Wardarski/AP
School & District Management Video 'Students Never Forget': Principals Call for Help After School Shootings
School leaders are lobbying Congress for more financial support for schools that experience gun violence.
2 min read
Forest High School students console one another after a school shooting at Forest High School Friday, April 20, 2018 in Ocala, Fla. One student shot another in the ankle at the high school and a suspect is in custody, authorities said Friday. The injured student was taken to a local hospital for treatment.
Forest High School students console one another after a school shooting at Forest High School Friday, April 20, 2018 in Ocala, Fla. One student shot another in the ankle at the high school and a suspect is in custody, authorities said Friday. The injured student was taken to a local hospital for treatment.
Doug Engle/Star-Banner via AP
School & District Management Opinion In School Leadership, Busy Is a Given. Chaos Is a Choice
There will never be enough time, money, or resources to solve every problem in education, so we must learn to operate within constraints.
Kate Hazarian
3 min read
Two hands attempt to hold chaos.
Vanessa Solis/Education Week via Canva
School & District Management How Sweltering Heat Disrupts Learning and What Schools Can Do
Extreme heat is becoming more common across the United States. Schools need to start preparing now.
5 min read
A boy cools off at a fountain during hot weather in Chicago, on June 16, 2024.
A boy cools off at a fountain during hot weather in Chicago, on June 16, 2024.
Nam Y. Huh/AP