No School Improvement Models Get Top Rating From AIR

By Debra Viadero — October 10, 2006 3 min read
  • Save to favorites
  • Print

Only 10 of the 18 most widely used school improvement programs for middle and high schools have “moderate” or “limited” evidence to show they work, and none deserves a top rating, a review by a Washington think tank concludes.

The review, released last week at a national conference in Houston, is the third in a series of consumer-style reports produced by the Comprehensive School Reform Quality Center at the American Institutes for Research. The federally financed reviews evaluate the research base for popular, prepackaged schoolwide-improvement designs and assign ratings based on their effectiveness and other characteristics.

“CSRQ Center Report on Middle and High School Comprehensive School Reform Models” is posted by the American Institutes for Research.

This time around, none of the models had a research track record robust enough to earn one of the nonprofit group’s two highest ratings—“very strong” or “strong.” Nonetheless, Steve Fleischman, the AIR vice president who oversees the project, characterized the results for secondary schools as encouraging.

“Part of this, I think, is that it’s really hard to do research at the high school or middle school level,” he said, “and some of the models are newer, so they haven’t had time to establish a research base.”

“But if we had done this a year and a half ago, there would have been even less evidence,” he added, describing 2005 as “a really good year for research.”

The four programs rated moderate for their effectiveness were: America’s Choice School Design, based in Washington; the School Development Program, the model developed by the Yale University psychologist James P. Comer; and Success for All—Middle Grades and Talent Development High School, both designed by researchers at Johns Hopkins University in Baltimore.

At the other end of the scale, no programs were found to lack any research evidence at all or to have a negative impact on student achievement. But the center’s analysts gave eight programs a rating of “zero,” meaning the studies found for those programs were not rigorous enough.

The center bases its standards on the federal government’s definition of “scientifically based research,” which tends to favor studies that use comparison and control groups.

Higher Standards

Among the zero-rated programs were well-established improvement models, such as High Schools That Work, of Atlanta; Accelerated Schools PLUS, of Storrs, Conn.; the Coalition of Essential Schools, based in Oakland, Calif.; and Modern Red Schoolhouse, of Nashville, Tenn.

How Effective Are They?

Four of 18 school improvement models the Comprehensive School Reform Quality Center evaluated were given a “moderate” rating for their effectiveness in raising student achievement:

• America’s Choice School Design, Washington
• School Development Program, New Haven, Conn.
• Success for All—Middle Grades, Baltimore
• Talent Development High Schools, Baltimore

The center gave “limited” effectiveness ratings to six other programs:

• Expeditionary Learning, Garrison, N.Y.
• First Things First, Toms River, N.J.
• Knowledge Is Power Program (KIPP), San Francisco
• Middle Start, New York City
• More Effective Schools, Kinderhook, N.Y.
• Project GRAD, Houston

Source: American Institutes for Research

“We knew we did not have the kind of research where you have control groups. We’ve never had the resources to do that, but we’d welcome it,” said Gene Bottoms, the senior vice president of the Southern Regional Education Board, which founded High Schools That Work 20 years ago.

Used in 1,100 schools nationwide, that model got a stronger rating from the think tank seven years ago, when it undertook a similar review of improvement programs. But the research group’s research-quality standards for the current project are tougher. Of the 1,500 effectiveness studies the center reviewed for the secondary school report, only 41 made the cut.

The reviewers, who concentrated on programs used in at least 40 sites and three or more states, also rated programs on other characteristics besides effectiveness. Analysts looked, for example, to see whether the programs are particularly effective with diverse student populations, whether they lead more families to become involved in schooling, and whether they provide needed services and support for schools.

The ratings for secondary school programs were not as high as they were for elementary schools, which were the focus of the center’s first report in 2005. (“Report Critiques Evidence on School Improvement Models,” Dec. 7, 2005.) That review, which has since been downloaded from the center’s Web site more than 50,000 times, gave “moderately strong” effectiveness to two programs and moderate ratings to five others.

Nevertheless, Mr. Fleischman said the secondary school report and an updated version of the elementary school report due out later this fall may be the center’s last. Its three-year, $4 million grant to vet the research on comprehensive school reform models runs out next month, and the U.S. Department of Education’s office of elementary and secondary education has no plans to extend it.

Mr. Fleischman said the AIR would continue to make the center’s reports available on its Web site.

Related Tags:

A version of this article appeared in the October 11, 2006 edition of Education Week as No School Improvement Models Get Top Rating From AIR


Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Building Teacher Capacity for Social-Emotional Learning
Set goals that support adult well-being and social-emotional learning: register today!

Content provided by Panorama
Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal 'A Snitch Line on Parents.' GOP Reps Grill AG Over Response to Threats on School Officials
Attorney General Merrick Garland said his effort is meant to address violent threats against school boards, not to stifle parents' dissent.
5 min read
LEFT: Attorney General Merrick Garland speaks during a House Judiciary Committee oversight hearing of the Department of Justice on Capitol Hill in Washington, Thursday, Oct. 21, 2021. RIGHT: Rep. Jim Jordan, R-Ohio, questions Attorney General Merrick Garland.
Attorney General Merrick Garland, left, speaks during a House Judiciary Committee oversight hearing of the U.S. Department of Justice on Capitol Hill on Thursday, questioned by Rep. Jim Jordan, R-Ohio, right, among others.
Greg Nash via AP, Andrew Harnik/AP
Federal School Boards, 'Domestic Terrorism,' and Free Speech: Inside the Debate
From critical race theory to COVID policy, the heat on schools has raised issues involving free speech and the safety of public officials.
13 min read
Brenda Stephens, a school board member with Orange County Public Schools in Hillsborough, N.C. has purchased a weapon and taken a concealed carry class over concerns for her personal safety.
Brenda Stephens, a school board member in Hillsborough, N.C., says board members face threats and bullying, an atmosphere far different from what she's encountered in years of board service.
Kate Medley for Education Week
Federal Senate Confirms Catherine Lhamon to Civil Rights Post; Kamala Harris Casts Decisive Vote
Joe Biden's controversial pick to lead the Education Department's office for civil rights held that job in the Obama administration.
2 min read
Catherine Lhamon, nominee to be assistant secretary for civil rights at the Department of Education, testifies during a Senate Health, Education Labor and Pensions Committee confirmation hearing in Dirksen Building on Tuesday, July 13, 2021.
Catherine Lhamon, then-nominee to be assistant secretary for civil rights at the U.S. Department of Education, testifies during a Senate Health, Education Labor and Pensions Committee confirmation hearing in July.
Tom Williams/CQ Roll Call via AP Images
Federal White House Outlines COVID-19 Vaccination Plans for Kids 5-11
The Biden administration will rely on schools, pharmacies, and pediatricians to help deliver the COVID-19 shots to younger children.
3 min read
Ticket number 937 sits on a COVID-19 vaccination at the drive-thru vaccination site in the Coweta County Fairgrounds on Jan. 14, 2021, in Newnan, Ga.
A ticket number sits on a COVID-19 vaccination at the drive-thru vaccination site in the Coweta County Fairgrounds in Newnan, Ga.
Curtis Compton/Atlanta Journal-Constitution via AP