School & District Management

Study Finds Charter Networks Give No Clear Edge on Results

November 08, 2011 5 min read

A new national study on the effectiveness of networks that operate charter schools finds overall that their middle school students’ test scores in reading, mathematics, science, and social studies aren’t significantly better than those of students in regular public schools.

The average results varied widely: Students in some charter networks managed three years of growth in two years; in others, students tested a year behind grade level after a year or two in the program.

The findings from the research group Mathematica and the Center on Reinventing Public Education at the University of Washington Bothell underscore the point that being run by a charter-management organization, or CMO, isn’t a predictor of an individual school’s or student’s success, and that CMOs cannot be lumped together as being effective or ineffective. Previous studies have shown the same about individual charters.

“I don’t think there’s any doubt some of the CMOs have done a great job,” said Thomas Toch, a Washington writer and policy expert who writes extensively about CMOs. “But it doesn’t mean that every CMO is going to be successful. That’s the clear message here. It suggests just how hard creating good new schools is, and how hard it is to scale networks with even very good schools.”

The study made public last week is part of a long-running project by Mathematica of Princeton, N.J., and the Center on Reinventing Public Education. It involved 40 CMOs with 292 schools in 14 states; all the management groups were nonprofits that controlled at least four schools and had at least four schools open in fall 2007.

The researchers focused on charter-management organizations to explore whether that model could be effective for scaling up the successes of individual charter schools. Charters are publicly funded but free of many rules governing regular public schools.

CMOs exist in part to address the unevenness in quality from charter to charter, said Robin Lake, the associate director of the Center on Reinventing Public Education. “There was a real question about, ‘Are CMOs helping to improve the quality of charters overall?’ ” And the answer, she said, is that they haven’t had a significant positive effect as a group.

The study also found that some practices associated with charter schools run by management organizations were particularly effective. Comprehensive behavior policies—including zero-tolerance, specific behavior codes with rewards and sanctions, and “contracts” with students or their parents about behavior—were identified as having a positive effect on students’ math and reading scores.

Also, charter-management organizations that provide intensive coaching of teachers, including frequent reviews of lesson plans and observation, appeared to boost student achievement.

Researchers also explored how quickly those organizations grew, whom the schools served, the resources they used, and what influenced their growth.

The report does not name the networks involved in the study. They were guaranteed anonymity for participating.

One finding from the 3½ -year-long project is that the CMOs serve a disproportionately large number of black, Hispanic, and low-income students—even more so than the districts in which they operate—but fewer students with disabilities and English-language learners.

The study was funded by the Bill & Melinda Gates Foundation and the Walton Family Foundation, with project-management assistance from the nonprofit NewSchools Venture Fund, which invests in charter schools and other educational innovations. (Gates also provides support for organizational capacity-building to Editorial Projects in Education, the nonprofit publisher of Education Week.)

Nationwide, about 130 CMOs serve some 250,000 students. They account for the operation of about one in five of the 5,000 charter schools in the country, an increase from 12 percent in 1999.

Joshua Haimson of Mathematica, the project director of the CMO studies, said examining the work of CMOs “allows us to answer two important questions: To what extent have CMOs been effective at expanding educational models, and how have they done that?”

Looking Under the Hood

To gauge charter networks’ success at promoting student achievement, the researchers compared their students’ performance with that of students at nearby district-run schools, and in some cases, with independently run charters. The researchers looked at test-score gains for individual students from a year before they entered the CMO schools to up to three years later and compared them with data from students who resembled them in nearby districts.

Of the 40 CMOs in the study, data from 22 were complete enough to be used in this portion of the report. Two years after enrolling, students at 11 of the 22 did significantly better in math while a third did significantly worse. In 10, students experienced positive effects in reading, while at six, there were negative results.

Because some CMOs were able to advance students three grade levels in two years, their methods have the potential to close achievement gaps, Mr. Haimson said. Larger networks of charter schools generally did better at improving student achievement than smaller ones.

In addition to behavior policy and teacher coaching, the researchers examined features often found in network charter schools, including their use of additional instructional time, performance-based pay for teachers, and frequent formative assessments.

At first, additional time offered at some CMO-run schools appeared to influence student performance, but digging deeper, the researchers determined it was the teacher coaching and behavior policies that were the actual drivers, Mr. Haimson said.

Schoolwide behavior strategies by some CMO schools include setting behavior standards and signed responsibility agreements, but schools also said they had more flexibility than district principals in defining the details of all behavior policies.

Teacher coaching included more-frequent observation of teachers and more feedback to teachers from those observations, as well as frequent reviews of teachers’ lesson plans. CMO schools were more likely than nearby public school systems to base teachers’ pay on student test scores and observations than on seniority and education.

While behavior policy and teacher coaching emerged as definitive ways to improve achievement, the other practices shouldn’t be discounted, said Ms. Lake. Nor should those strategies be considered surefire.

“The takeaway shouldn’t be, if you just plug in a good behavior policy, you’re going to see good results,” Ms. Lake said.

The study also looked at how much CMOs spend per student compared with regular schools. Spending in the charters studied ranged from $5,000 to $20,000 per student a year, including public and private money.

The mixed results lead to a larger conclusion about school reform, said Mr. Toch, also a former reporter and editor for Education Week in the 1980s. “CMOs as a strategy are only one piece of the school improvement puzzle,” he said. “I don’t think we can expect to see thousands of truly game-changing schools from the CMO movement,” even though some such networks have shown success.

Nirvi Shah, Writer contributed to this article.
A version of this article appeared in the November 09, 2011 edition of Education Week as Study Finds Charter Networks Give No Clear Edge on Results

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Building Leadership Excellence Through Instructional Coaching
Join this webinar for a discussion on instructional coaching and ways you can link your implement or build on your program.
Content provided by Whetstone Education/SchoolMint
Teaching Webinar Tips for Better Hybrid Learning: Ask the Experts What Works
Register and ask your questions about hybrid learning to our expert panel.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Families & the Community Webinar
Family Engagement for Student Success With Dr. Karen Mapp
Register for this free webinar to learn how to empower and engage families for student success featuring Karen L. Mapp.
Content provided by Panorama Education & PowerMyLearning

EdWeek Top School Jobs

DevOps Engineer
Portland, OR, US
Northwest Evaluation Association
User Experience Analyst
Portland, OR, US
Northwest Evaluation Association
Senior Business Analyst - 12 Month Contract
Portland, OR, US
Northwest Evaluation Association
Senior Director Marketing
Philadelphia, Pennsylvania, United States
Camelot Education

Read Next

School & District Management One District's COVID-19 Journey: A Year of Upheaval and Unexpected Insights
The Everett, Wash., school district was among the first in the nation shuttered by the pandemic. Educators and parents share and reflect.
9 min read
School & District Management The Key to School-Based COVID-19 Testing: Cooperation of Parents and Communities
As schools launch broad testing to track cases of COVID-19, the success of their efforts relies on addressing the concerns of all concerned.
7 min read
Katie Ramirez, left, watches as her mother, Claudia Campos, swabs the mouth of her sister, Hailey, for a COVID-19 test at a testing site in Los Angeles on Dec. 9, 2020.
Katie Ramirez, left, watches as her mother, Claudia Campos, swabs the mouth of her sister, Hailey, for a COVID-19 test at a testing site in Los Angeles.
Jae C. Hong/AP
School & District Management Interactive A Year of COVID-19: What It Looked Like for Schools
This timeline offers a look at how a full year of living and learning during the COVID-19 pandemic unfolded.
Education Week Staff
13 min read
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Anthony Vazquez/Chicago Sun-Times via AP
School & District Management Most Principals, District Leaders Predict Their Schools Will Be Fully In-Person This Fall
EdWeek Research Center surveys track the growing trend to get more students back in school buildings as soon as possible.
5 min read
Assistant Principal Janette Van Gelderen, left, welcomes students at Newhall Elementary in Santa Clarita, Calif on Feb. 25, 2021. California's public schools could get $6.6 billion from the state Legislature if they return to in-person instruction by the end of March, according to a new agreement announced Monday, March 1, 2021, between Gov. Gavin Newsom and the state's legislative leaders.
Assistant Principal Janette Van Gelderen, left, welcomes students at Newhall Elementary in Santa Clarita, Calif., last month. California's public schools could get $6.6 billion from the state if they return to in-person instruction by the end of March.
Marcio Jose Sanchez/AP