Study Finds Research on Management Groups Lacking

By Debra Viadero — May 02, 2006 4 min read
  • Save to favorites
  • Print

Some of the most prominent educational management organizations hired to run schools have a thin, or nonexistent, research base to prove they work, according to a consumer guide released last week by a Washington-based research group.

“CSRQ Center Report on Education Service Providers,” April 2006, is posted by the Comprehensive School Reform Quality Center.

Under a three-year U.S. Department of Education grant, researchers from the American Institutes for Research weighed the research evidence on seven widely used educational management organizations, or EMOs. Such organizations, also known as educational service providers, are outside nonprofit or for-profit operators of public schools.

Only one of the EMOs—Edison Schools Inc., a New York City-based for-profit company—could produce enough high-quality studies to show that its schools can improve student achievement. It earned a “moderate” effectiveness rating from the research group. But no providers got AIR’s top two ratings: “moderately strong” and “very strong.”

John E. Chubb, the chief education officer for Edison, which runs 157 schools around the country, praised the report but called for caution in interpreting its findings.

“Even though some of these organizations have been around a while, this [outside management] is still an activity that is in its infancy,” he said. “This [report] will be very useful in the hands of informed policymakers, but another reaction to this is that policymakers could say, ‘There’s no evidence that this works, so let’s move on.’ ”

No Search for ‘Best Buys’

For their part, the researchers noted that their aim was not to single out “best buys” among educational service providers. They sought instead to evaluate information that educators and policymakers could use to “clarify their options” for improving schools.

Besides rating organizations on the effectiveness of learning at their schools, the guide judges the providers on a range of other factors, including whether they provide schools with professional development for staff members and whether their instructional programs are based on research.

Weighing the Research

A new report provides consumer-type reviews of seven educational management organizations.

*Click image to see the full chart.

Click to Enlarge: Weighing the Research

SOURCE: AIR’s Comprehensive School Reform Quality Center

Despite its “moderate” effectiveness rating, for instance, Edison got a “very strong” rating for providing professional development, acquiring a track record of successful implementation, and linking practices to research.

Overall, the researchers gave no providers a “limited” rating for success at improving student achievement. But four for-profit groups got ratings of “zero,” meaning that the studies found on those organizations weren’t rigorous enough to meet the researchers’ standards. Those organizations are: the Leona Group LLC, based in Phoenix, Ariz., and East Lansing, Mich.; Mosaica Education, of Atlanta; National Heritage Academies, of Grand Rapids, Mich.; and White Hat Management, based in Akron, Ohio.

Two other groups—Imagine Schools, a nonprofit school management provider based in Arlington, Va., and SABIS Educational Systems, an Eden Prairie, Minn.-based for-profit company—got no ratings because the researchers could not find any studies to show whether their models translated into learning gains.

No companies were found to negatively affect learning.

Tough Standards

The seven models that were examined operate in 350 schools—about 60 percent to 65 percent of all the schools across the country that use such outside groups.

One provider, the Leona Group, took issue with how it was characterized by the report. In a letter appended to the study, the organization said its zero rating was “misleading” because the company does not mandate a single curricular approach for all of its schools.

Michael J. Connell, Mosaica’s chief executive officer, and Casey Carter, the president of National Heritage Academies, both said that their research supports the effectiveness of their programs, though their studies did not fit the researchers’ methodology.

The guide is the second consumer-style effort produced by AIR’s newly formed Comprehensive School Reform Quality (CSRQ) Center. Last year, the center evaluated studies on 22 schoolwide improvement approaches used in elementary schools. That effort yielded more-promising results: two programs with “moderately strong” evidence of learning success, five programs with a “moderate” rating, and eight with a “limited” achievement impact, among other findings.

One reason for the thinner research showing among educational service providers, the authors said, was that four of the seven did not respond to the researchers’ information requests.

“It’s troubling that a majority of the providers eschewed their responsibility to public accountability,” said Nancy Van Meter, a specialist on privatization for the American Federation of Teachers, which is critical of outside management of public schools.

But AIR researchers noted that they set tough standards for judging research quality for both reports. Of the 940 studies the center screened on educational management organizations, for instance, only nine made the final cut.

Steve Fleischman, the AIR vice president who oversaw the project, said the reform-quality center’s standards closely track the description of “scientifically based research” set in the federal No Child Left Behind Act, which tends to favor experiments akin to those used for medical studies.

“We are in an era when NCLB asks low-performing schools and asks anybody interested in school improvement to do so based on scientific research,” he said.

A version of this article appeared in the May 03, 2006 edition of Education Week as Study Finds Research on Management Groups Lacking


Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Top Federal Adviser on Puerto Rico's Schools Declares: 'We Have to Build Trust'
Chris Soto heads an Education Department team providing technical assistance and support for the U.S. territory's public schools.
4 min read
Martin G. Brumbaugh School kindergarten teacher Nydsy Santiago teaches her students under a gazebo at a municipal athletic park in Santa Isabel, Puerto Rico, on Feb. 4, 2021.
Martin G. Brumbaugh School kindergarten teacher Nydsy Santiago teaches her students under a gazebo at a municipal athletic park in Santa Isabel, Puerto Rico, on Feb. 4, 2021.
Carlos Giusti/AP
Federal Schools Could Count Nonbinary Students Under Biden Proposal
The Civil Rights Data Collection for this school year could also revive questions about inexperienced teachers and preschool discipline.
6 min read
Image of a form with male and female checkboxes.
Federal 'Parents' Bill of Rights' Underscores Furor Over Curriculum and Transparency in Schools
U.S. Sen. Josh Hawley's bill highlights how education issues like critical race theory will likely stay in the national political spotlight.
7 min read
Sen. Josh Hawley, R-Mo., speaks during a Senate Armed Services Committee hearing on the conclusion of military operations in Afghanistan and plans for future counterterrorism operations, Tuesday, Sept. 28, 2021, on Capitol Hill in Washington.
U.S. Sen. Josh Hawley, R-Mo., says "it's time to give control back to parents, not woke bureaucrats."
Patrick Semansky/AP
Federal Opinion It’s Not Just the NSBA That’s Out of Touch. There’s a Bigger Problem
Those who influence educational policy or practice would do well to care about what parents and the public actually want.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty