Federal

No School Improvement Models Get Top Rating From AIR

By Debra Viadero — October 10, 2006 3 min read

Only 10 of the 18 most widely used school improvement programs for middle and high schools have “moderate” or “limited” evidence to show they work, and none deserves a top rating, a review by a Washington think tank concludes.

The review, released last week at a national conference in Houston, is the third in a series of consumer-style reports produced by the Comprehensive School Reform Quality Center at the American Institutes for Research. The federally financed reviews evaluate the research base for popular, prepackaged schoolwide-improvement designs and assign ratings based on their effectiveness and other characteristics.

“CSRQ Center Report on Middle and High School Comprehensive School Reform Models” is posted by the American Institutes for Research.

This time around, none of the models had a research track record robust enough to earn one of the nonprofit group’s two highest ratings—“very strong” or “strong.” Nonetheless, Steve Fleischman, the AIR vice president who oversees the project, characterized the results for secondary schools as encouraging.

“Part of this, I think, is that it’s really hard to do research at the high school or middle school level,” he said, “and some of the models are newer, so they haven’t had time to establish a research base.”

“But if we had done this a year and a half ago, there would have been even less evidence,” he added, describing 2005 as “a really good year for research.”

The four programs rated moderate for their effectiveness were: America’s Choice School Design, based in Washington; the School Development Program, the model developed by the Yale University psychologist James P. Comer; and Success for All—Middle Grades and Talent Development High School, both designed by researchers at Johns Hopkins University in Baltimore.

At the other end of the scale, no programs were found to lack any research evidence at all or to have a negative impact on student achievement. But the center’s analysts gave eight programs a rating of “zero,” meaning the studies found for those programs were not rigorous enough.

The center bases its standards on the federal government’s definition of “scientifically based research,” which tends to favor studies that use comparison and control groups.

Higher Standards

Among the zero-rated programs were well-established improvement models, such as High Schools That Work, of Atlanta; Accelerated Schools PLUS, of Storrs, Conn.; the Coalition of Essential Schools, based in Oakland, Calif.; and Modern Red Schoolhouse, of Nashville, Tenn.

How Effective Are They?

Four of 18 school improvement models the Comprehensive School Reform Quality Center evaluated were given a “moderate” rating for their effectiveness in raising student achievement:

• America’s Choice School Design, Washington
• School Development Program, New Haven, Conn.
• Success for All—Middle Grades, Baltimore
• Talent Development High Schools, Baltimore

The center gave “limited” effectiveness ratings to six other programs:

• Expeditionary Learning, Garrison, N.Y.
• First Things First, Toms River, N.J.
• Knowledge Is Power Program (KIPP), San Francisco
• Middle Start, New York City
• More Effective Schools, Kinderhook, N.Y.
• Project GRAD, Houston

Source: American Institutes for Research

“We knew we did not have the kind of research where you have control groups. We’ve never had the resources to do that, but we’d welcome it,” said Gene Bottoms, the senior vice president of the Southern Regional Education Board, which founded High Schools That Work 20 years ago.

Used in 1,100 schools nationwide, that model got a stronger rating from the think tank seven years ago, when it undertook a similar review of improvement programs. But the research group’s research-quality standards for the current project are tougher. Of the 1,500 effectiveness studies the center reviewed for the secondary school report, only 41 made the cut.

The reviewers, who concentrated on programs used in at least 40 sites and three or more states, also rated programs on other characteristics besides effectiveness. Analysts looked, for example, to see whether the programs are particularly effective with diverse student populations, whether they lead more families to become involved in schooling, and whether they provide needed services and support for schools.

The ratings for secondary school programs were not as high as they were for elementary schools, which were the focus of the center’s first report in 2005. (“Report Critiques Evidence on School Improvement Models,” Dec. 7, 2005.) That review, which has since been downloaded from the center’s Web site more than 50,000 times, gave “moderately strong” effectiveness to two programs and moderate ratings to five others.

Nevertheless, Mr. Fleischman said the secondary school report and an updated version of the elementary school report due out later this fall may be the center’s last. Its three-year, $4 million grant to vet the research on comprehensive school reform models runs out next month, and the U.S. Department of Education’s office of elementary and secondary education has no plans to extend it.

Mr. Fleischman said the AIR would continue to make the center’s reports available on its Web site.

Related Tags:

A version of this article appeared in the October 11, 2006 edition of Education Week as No School Improvement Models Get Top Rating From AIR

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Building Leadership Excellence Through Instructional Coaching
Join this webinar for a discussion on instructional coaching and ways you can link your implement or build on your program.
Content provided by Whetstone Education/SchoolMint
Teaching Webinar Tips for Better Hybrid Learning: Ask the Experts What Works
Register and ask your questions about hybrid learning to our expert panel.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Families & the Community Webinar
Family Engagement for Student Success With Dr. Karen Mapp
Register for this free webinar to learn how to empower and engage families for student success featuring Karen L. Mapp.
Content provided by Panorama Education & PowerMyLearning

EdWeek Top School Jobs

[2021-2022] Founding Middle School Academic Dean
New York, NY, US
DREAM Charter School
Hiring Bilingual and Special Education Teachers NOW!
Newark, New Jersey
Newark Public Schools
DevOps Engineer
Portland, OR, US
Northwest Evaluation Association
Senior Business Analyst - 12 Month Contract
Portland, OR, US
Northwest Evaluation Association

Read Next

Federal Biden Announces Goal to Get Educators the COVID-19 Vaccine This Month
President Joe Biden pushes states to get educators at least one dose by the end of March to help schools resume in-person learning.
4 min read
John Battle High School teacher Jennifer Daniel receives her COVID-19 vaccine on Jan. 11, 2021. Teachers received their first vaccine during an all-day event at the Virginia Highlands Higher Education Center in Abingdon, Va.
John Battle High School teacher Jennifer Daniel receives her COVID-19 vaccine on Jan. 11, 2021. Teachers received their first vaccine during an all-day event at the Virginia Highlands Higher Education Center in Abingdon, Va.
David Crigger/Bristol Herald Courier via AP
Federal Explainer Miguel Cardona, U.S. Secretary of Education: Background and Achievements
Background and highlights of Miguel Cardona's tenure as the twelfth U.S. Secretary of Education.
Education Week Library
2 min read
Miguel Cardona, President-elect Joe Biden's nominee for Secretary of Education, speaks after being introduced at The Queen Theater in Wilmington, Del., on Dec. 23, 2020.
Miguel Cardona, U.S. Secretary of Education, speaks after being put forward for the position by then-President-elect Joe Biden in December 2020.
Carolyn Kaster/AP
Federal Senate Confirms Miguel Cardona as Education Secretary
The former Connecticut education commissioner got his start as an elementary school teacher and was a principal and school administrator.
2 min read
Miguel Cardona, President-elect Joe Biden's nominee for Secretary of Education, speaks after being introduced at The Queen Theater in Wilmington, Del., Wednesday, Dec. 23, 2020.
Miguel Cardona was confirmed by the Senate to serve as U.S. Secretary of Education. The former Connecticut education commissioner has worked as a teacher, principal, and district administrator.
Carolyn Kaster/AP
Federal Biden Legal Team Steps Back From Trump Stance on Transgender Female Sports Participation
The Education Department's office for civil rights pulls a letter that said Connecticut's transgender-inclusive policy violates Title IX.
4 min read
Bloomfield High School transgender athlete Terry Miller, second from left, wins the final of the 55-meter dash over transgender athlete Andraya Yearwood, far left, and other runners in the Connecticut girls Class S indoor track meet at Hillhouse High School in New Haven, Conn on Feb. 7, 2019. Transgender athletes are getting an ally in the White House next week as they seek to participate as their identified gender in high school and college sports. Attorneys on both sides say they expect President-elect Joe Biden’s Department of Education will switch sides in legal battles that could go a long way in determining whether transgender athletes are treated by the sex on their birth certificates or by how they identify.
Bloomfield High School transgender athlete Terry Miller, second from left, wins over transgender athlete Andraya Yearwood, far left, and other runners in an event in New Haven, Conn. The two transgender athletes are at the center of a legal fight in Connecticut over the participation of transgender female athletes in girls' or women's sports.
Pat Eaton-Robb/AP