Federal

No School Improvement Models Get Top Rating From AIR

By Debra Viadero — October 10, 2006 3 min read
  • Save to favorites
  • Print

Only 10 of the 18 most widely used school improvement programs for middle and high schools have “moderate” or “limited” evidence to show they work, and none deserves a top rating, a review by a Washington think tank concludes.

The review, released last week at a national conference in Houston, is the third in a series of consumer-style reports produced by the Comprehensive School Reform Quality Center at the American Institutes for Research. The federally financed reviews evaluate the research base for popular, prepackaged schoolwide-improvement designs and assign ratings based on their effectiveness and other characteristics.

“CSRQ Center Report on Middle and High School Comprehensive School Reform Models” is posted by the American Institutes for Research.

This time around, none of the models had a research track record robust enough to earn one of the nonprofit group’s two highest ratings—“very strong” or “strong.” Nonetheless, Steve Fleischman, the AIR vice president who oversees the project, characterized the results for secondary schools as encouraging.

“Part of this, I think, is that it’s really hard to do research at the high school or middle school level,” he said, “and some of the models are newer, so they haven’t had time to establish a research base.”

“But if we had done this a year and a half ago, there would have been even less evidence,” he added, describing 2005 as “a really good year for research.”

The four programs rated moderate for their effectiveness were: America’s Choice School Design, based in Washington; the School Development Program, the model developed by the Yale University psychologist James P. Comer; and Success for All—Middle Grades and Talent Development High School, both designed by researchers at Johns Hopkins University in Baltimore.

At the other end of the scale, no programs were found to lack any research evidence at all or to have a negative impact on student achievement. But the center’s analysts gave eight programs a rating of “zero,” meaning the studies found for those programs were not rigorous enough.

The center bases its standards on the federal government’s definition of “scientifically based research,” which tends to favor studies that use comparison and control groups.

Higher Standards

Among the zero-rated programs were well-established improvement models, such as High Schools That Work, of Atlanta; Accelerated Schools PLUS, of Storrs, Conn.; the Coalition of Essential Schools, based in Oakland, Calif.; and Modern Red Schoolhouse, of Nashville, Tenn.

How Effective Are They?

Four of 18 school improvement models the Comprehensive School Reform Quality Center evaluated were given a “moderate” rating for their effectiveness in raising student achievement:

• America’s Choice School Design, Washington
• School Development Program, New Haven, Conn.
• Success for All—Middle Grades, Baltimore
• Talent Development High Schools, Baltimore

The center gave “limited” effectiveness ratings to six other programs:

• Expeditionary Learning, Garrison, N.Y.
• First Things First, Toms River, N.J.
• Knowledge Is Power Program (KIPP), San Francisco
• Middle Start, New York City
• More Effective Schools, Kinderhook, N.Y.
• Project GRAD, Houston

Source: American Institutes for Research

“We knew we did not have the kind of research where you have control groups. We’ve never had the resources to do that, but we’d welcome it,” said Gene Bottoms, the senior vice president of the Southern Regional Education Board, which founded High Schools That Work 20 years ago.

Used in 1,100 schools nationwide, that model got a stronger rating from the think tank seven years ago, when it undertook a similar review of improvement programs. But the research group’s research-quality standards for the current project are tougher. Of the 1,500 effectiveness studies the center reviewed for the secondary school report, only 41 made the cut.

The reviewers, who concentrated on programs used in at least 40 sites and three or more states, also rated programs on other characteristics besides effectiveness. Analysts looked, for example, to see whether the programs are particularly effective with diverse student populations, whether they lead more families to become involved in schooling, and whether they provide needed services and support for schools.

The ratings for secondary school programs were not as high as they were for elementary schools, which were the focus of the center’s first report in 2005. (“Report Critiques Evidence on School Improvement Models,” Dec. 7, 2005.) That review, which has since been downloaded from the center’s Web site more than 50,000 times, gave “moderately strong” effectiveness to two programs and moderate ratings to five others.

Nevertheless, Mr. Fleischman said the secondary school report and an updated version of the elementary school report due out later this fall may be the center’s last. Its three-year, $4 million grant to vet the research on comprehensive school reform models runs out next month, and the U.S. Department of Education’s office of elementary and secondary education has no plans to extend it.

Mr. Fleischman said the AIR would continue to make the center’s reports available on its Web site.

Related Tags:

A version of this article appeared in the October 11, 2006 edition of Education Week as No School Improvement Models Get Top Rating From AIR

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
Teaching Profession Live Online Discussion What Have We Learned From Teachers During the Pandemic?
University of California, Santa Cruz, researcher Lora Bartlett and her colleagues spent months studying how the pandemic affected classroom teachers. We will discuss the takeaways from her research not only for teachers, but also for
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Trauma-Informed Practices & the Construction of the Deep Reading Brain
Join Ryan Lee-James, Ph.D. CCC-SLP, director of the Rollins Center for Language and Literacy, with Renée Boynton-Jarrett, MD, ScD., Vital Village Community Engagement Network; Neena McConnico, Ph.D, LMHC, Child Witness to Violence Project; and Sondra
Content provided by Rollins Center

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal How a Big Federal Spending Package Could Affect School Meals and Student Poverty Counts
Legislation to expand access to free school meals highlights a persistent concern: how to improve the ways we identify students in poverty.
6 min read
Food service assistant Brenda Bartee, rear, gives students breakfast, Tuesday, Aug. 10, 2021, during the first day of school at Washington Elementary School in Riviera Beach, Fla.
Food service assistant Brenda Bartee, rear, gives students breakfast, last month on the first day of school at Washington Elementary School in Riviera Beach, Fla.
Wilfredo Lee/AP
Federal Feds to Probe Whether Texas Ban on School Mask Mandates Violates Disability Rights Laws
The Education Department has already opened investigations in six other states that ban universal school mask requirements.
2 min read
A staff member holds the door open for kids on the first day of school at Goodwin Frazier Elementary School in New Braunfels, Texas on Tuesday, Aug. 25, 2020.
A staff member holds the door open at Goodwin Frazier Elementary School in New Braunfels, Texas in 2020. This year, Texas has prohibited school districts from requiring all students to wear masks.
Mikala Compton/Herald-Zeitung via AP
Federal New Federal Team to Work on Puerto Rico School Improvement, Oversight
The Puerto Rico Education Sustainability Team will focus on creating better learning environments and improving financial management.
3 min read
Secretary of Education Miguel Cardona visits the Emilio Delgado School in Corozal on June 30, 2021 during a visit to Puerto Rico.
Secretary of Education Miguel Cardona visits the Emilio Delgado School in Corozal on June 30, 2021 during a visit to Puerto Rico.
Teresa Canino Rivera/GDA via AP
Federal Pandemic Tests Limits of Cardona's Collaborative Approach as Education Secretary
He's sought the image of a veteran educator among former peers, but COVID has forced him to take a tough stance toward some state leaders.
10 min read
Education Secretary Miguel Cardona and Schools Chancellor Meisha Porter speak to Mia Arias, 10, during their visit to P.S. 5 Port Morris, a Bronx elementary school, Tuesday, Aug. 17, 2021 in New York.
U.S. Secretary of Education Miguel Cardona and Schools Chancellor Meisha Porter speak to Mia Arias, 10, during a visit to P.S. 5 Port Morris, a Bronx elementary school, last month.
Brittainy Newman/AP