Some of the most prominent educational management organizations hired to run schools have a thin, or nonexistent, research base to prove they work, according to a consumer guide released last week by a Washington-based research group.
“CSRQ Center Report on Education Service Providers,” April 2006, is posted by the Comprehensive School Reform Quality Center.
Under a three-year U.S. Department of Education grant, researchers from the American Institutes for Research weighed the research evidence on seven widely used educational management organizations, or EMOs. Such organizations, also known as educational service providers, are outside nonprofit or for-profit operators of public schools.
Only one of the EMOs—Edison Schools Inc., a New York City-based for-profit company—could produce enough high-quality studies to show that its schools can improve student achievement. It earned a “moderate” effectiveness rating from the research group. But no providers got AIR’s top two ratings: “moderately strong” and “very strong.”
John E. Chubb, the chief education officer for Edison, which runs 157 schools around the country, praised the report but called for caution in interpreting its findings.
“Even though some of these organizations have been around a while, this [outside management] is still an activity that is in its infancy,” he said. “This [report] will be very useful in the hands of informed policymakers, but another reaction to this is that policymakers could say, ‘There’s no evidence that this works, so let’s move on.’ ”
No Search for ‘Best Buys’
For their part, the researchers noted that their aim was not to single out “best buys” among educational service providers. They sought instead to evaluate information that educators and policymakers could use to “clarify their options” for improving schools.
Besides rating organizations on the effectiveness of learning at their schools, the guide judges the providers on a range of other factors, including whether they provide schools with professional development for staff members and whether their instructional programs are based on research.
A new report provides consumer-type reviews of seven educational management organizations.
*Click image to see the full chart.
SOURCE: AIR’s Comprehensive School Reform Quality Center
Despite its “moderate” effectiveness rating, for instance, Edison got a “very strong” rating for providing professional development, acquiring a track record of successful implementation, and linking practices to research.
Overall, the researchers gave no providers a “limited” rating for success at improving student achievement. But four for-profit groups got ratings of “zero,” meaning that the studies found on those organizations weren’t rigorous enough to meet the researchers’ standards. Those organizations are: the Leona Group LLC, based in Phoenix, Ariz., and East Lansing, Mich.; Mosaica Education, of Atlanta; National Heritage Academies, of Grand Rapids, Mich.; and White Hat Management, based in Akron, Ohio.
Two other groups—Imagine Schools, a nonprofit school management provider based in Arlington, Va., and SABIS Educational Systems, an Eden Prairie, Minn.-based for-profit company—got no ratings because the researchers could not find any studies to show whether their models translated into learning gains.
No companies were found to negatively affect learning.
Tough Standards
The seven models that were examined operate in 350 schools—about 60 percent to 65 percent of all the schools across the country that use such outside groups.
One provider, the Leona Group, took issue with how it was characterized by the report. In a letter appended to the study, the organization said its zero rating was “misleading” because the company does not mandate a single curricular approach for all of its schools.
Michael J. Connell, Mosaica’s chief executive officer, and Casey Carter, the president of National Heritage Academies, both said that their research supports the effectiveness of their programs, though their studies did not fit the researchers’ methodology.
The guide is the second consumer-style effort produced by AIR’s newly formed Comprehensive School Reform Quality (CSRQ) Center. Last year, the center evaluated studies on 22 schoolwide improvement approaches used in elementary schools. That effort yielded more-promising results: two programs with “moderately strong” evidence of learning success, five programs with a “moderate” rating, and eight with a “limited” achievement impact, among other findings.
One reason for the thinner research showing among educational service providers, the authors said, was that four of the seven did not respond to the researchers’ information requests.
“It’s troubling that a majority of the providers eschewed their responsibility to public accountability,” said Nancy Van Meter, a specialist on privatization for the American Federation of Teachers, which is critical of outside management of public schools.
But AIR researchers noted that they set tough standards for judging research quality for both reports. Of the 940 studies the center screened on educational management organizations, for instance, only nine made the final cut.
Steve Fleischman, the AIR vice president who oversaw the project, said the reform-quality center’s standards closely track the description of “scientifically based research” set in the federal No Child Left Behind Act, which tends to favor experiments akin to those used for medical studies.
“We are in an era when NCLB asks low-performing schools and asks anybody interested in school improvement to do so based on scientific research,” he said.
