School & District Management

Academic Gains Vary Widely for Charter Networks

November 04, 2011 7 min read

A new national study on the effectiveness of networks that operate charter schools finds overall that their middle school students’ test scores in reading, mathematics, science, and social studies aren’t significantly better than those of students in regular public schools.

The average results varied widely: Students in some charter networks managed three years of growth in two years; in others, students tested a year behind grade level after a year or two in the program.

The findings from the research group Mathematica and the Center on Reinventing Public Education at the University of Washington Bothell underscore the point that being run by a charter-management organization, or CMO, isn’t a predictor of an individual school’s or student’s success, and that CMOs cannot be lumped together as being effective or ineffective. Previous studies have shown the same about individual charters.

“I don’t think there’s any doubt some of the CMOs have done a great job,” said Thomas Toch, a Washington writer and policy expert who writes extensively about CMOs. “But it doesn’t mean that every CMO is going to be successful. That’s the clear message here. It suggests just how hard creating good new schools is, and how hard it is to scale networks with even very good schools.”

The study made public last week is part of long-running project by Mathematica, of Princeton, N.J., and the Center on Reinventing Public Education. It involved 40 CMOs with 292 schools in 14 states; all of the management groups were nonprofits that controlled at least four schools and had at least four schools open in the fall of 2007.

The researchers focused on charter-management organizations to explore whether that model could be effective for scaling up the successes of individual charter schools. Charters receive public funding but are free of many of the rules governing regular public schools.

CMOs exist in part to address the unevenness in quality from charter to charter, said Robin Lake, the associate director of the Center on Reinventing Public Education. “There was a real question about ‘Are CMOs helping to improve quality of charters overall?’ ”

And the answer, she said, is that they haven’t had a significant positive effect as a group. Being a CMO is not a guarantee of quality. “You never can escape the work of replicating what works.”

The study also found that some practices associated with charter schools run by management organizations were particularly effective and others were less so. Comprehensive behavior policies—including zero-tolerance policies, specific behavior codes with rewards and sanctions, and “contracts” with students or their parents about behavior—were identified as having a positive effect on students’ math and reading scores.

Also, charter-management organizations that provide intensive coaching of teachers, including frequent reviews of lesson plans and observation, appeared to boost student achievement.

Researchers also explored how quickly those organizations grew, whom the schools served, the resources they used, and what influenced their growth.

The report does not disclose the names of the networks involved in the study. They were guaranteed anonymity for participating.

One finding from the three-and-a-half-year-long project is that the CMOs serve a disproportionately large number of black, Hispanic, and low-income students—even more so than the districts in which they operate—but fewer students with disabilities and English-language learners.

The study was funded by the Bill & Melinda Gates Foundation and the Walton Family Foundation, with project management assistance from the nonprofit NewSchools Venture Fund, which invests in charter schools and other educational innovations. (Gates also provides support for organizational capacity-building to Editorial Projects in Education, the nonprofit publisher of Education Week.)

Nationwide, about 130 CMOs serve some 250,000 students. They account for the operation of about one in five of the 5,000 charter schools in the country, an increase from 12 percent in 1999.

Joshua Haimson of Mathematica, the project director of the CMO studies, said examining the work of CMOs “allows us to answer two important questions: To what extent have CMOs been effective at expanding educational models, and how have they done that?”

To gauge charter networks’ success at promoting student achievement, the researchers compared charter school students’ performance with the achievement of students at nearby district-run schools, and in some cases, with independently run charters. The researchers looked at gains in the test scores of individual students from a year before they entered the CMO schools to up to three years after they entered school and compared those gains to data from a group of students who resembled them in the school districts nearby.

Of the 40 CMOs in the study, data from 22 were complete enough to be used in this portion of the report. Two years after students enrolled, students at 11 of the 22 did significantly better in math while a third did significantly worse. In 10, students had positive effects in reading, while at six, there were negative results.

Because some CMOs were able to advance students three grade levels in two years, their methods have the potential to close achievement gaps, Mr. Haimson said. Larger networks of charter schools generally did better at improving student achievement than smaller ones.

Looking Under the Hood

In addition to behavior policy and teacher coaching, the researchers examined features often found in network charter schools, including their use of additional instructional time, performance-based pay for teachers, and the use of frequent formative assessments.

At first, additional time offered at some CMO-run schools appeared to influence student performance, but digging deeper, the researchers determined it was the teacher coaching and behavior policies that were the actual drivers, Mr. Hamison said.

Schoolwide behavior strategies by some CMO schools include setting behavior standards and signed responsibility agreements, but schools also said they had more flexibility than district principals in defining the details of all behavior policies.

Teacher coaching included more-frequent observation of teachers and more feedback to teachers from those observations, as well as frequent reviews of teachers’ lesson plans. CMO schools were more likely than nearby public school systems to base teachers’ pay on student test scores and teacher observations than on seniority and education.

While behavior policy and teacher coaching emerged as definitive ways to improve student achievement, the other practices shouldn’t be discounted, said Ms. Lake. Nor should those strategies be considered surefire on their own.

“The takeaway shouldn’t be, if you just plug in a good behavior policy, you’re going to see good results,” Ms. Lake said.

Something researchers couldn’t measure was the power of the high expectations for students’ behavior at some schools. The “no excuses” model, associated in particular with the Knowledge Is Power Program, or KIPP, network of charter schools, seemed to matter, Ms. Lake said.

Spending Varies

The study also looked at how much CMOs spend per student compared with regular schools. Spending in the charters studied ranged from $5,000 per student to $20,000 a year, including public money and funds from private philanthropy, variations widely attributed to state funding formulas. CMO schools tend to be much smaller than schools in their host districts, and their pupil-teacher ratios are marginally lower.

“We need to collect the data to see what’s going well and what’s not,” said James Willcox, the chief executive officer of Aspire Public Schools, a California network of charter schools with more than 12,000 students in kindergarten through 12th grade. “The big finding here is that the practices of CMOs are really powerful: The highest-performing CMOs are getting three years of gains in only two years of school.” Although the study does not disclose whether his schools were involved in the study, they meet the researchers’ criteria.

The researchers plan to do a similar report on high schools in charter networks and produce a series of briefs on best practices for CMOs when their work is complete.

The mixed results of the research provide an important lesson for school boards and other agencies that approve new charter schools, said Alex Medler, the vice president of research and evaluation for the Chicago-based National Association of Charter School Authorizers, who said he welcomed the report.

“You don’t want all CMOs if you’re an authorizer,” he said. “There are some that are really good. A few of them aren’t very good.”

That leads to a larger conclusion about school reform, said Mr. Toch, who is also a former reporter and editor for Education Week.

“CMOs as a strategy are only one piece of the school improvement puzzle,” he said. “I don’t think we can expect to see thousands of truly game-changing schools from the CMO movement,” even though such networks as KIPP, Aspire, and Achievement First are growing and have shown success.

Mr. Toch said it may be more useful to consider more projects like one the Gates Foundation is working on to connect high-achieving CMOs with school districts to share best practices: “That offers an opportunity to scale things up.”

Nirvi Shah, Writer contributed to this article.
A version of this article appeared in the November 09, 2011 edition of Education Week as Academic Gains Vary Widely for Charter Networks

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Recruitment & Retention Webinar
Recruiting and Retaining a More Diverse Teaching Workforce
We discuss the importance of workforce diversity and learn strategies to recruit and retain teachers from diverse backgrounds.
Content provided by EdWeek Top School Jobs
Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children

EdWeek Top School Jobs

Principal
Meredith, New Hampshire
Inter-Lakes School District
Elementary Principal
Washington State
Wenatchee School District
Principal
Meredith, New Hampshire
Inter-Lakes School District
Elementary Principal
Washington State
Wenatchee School District

Read Next

School & District Management San Francisco School Board Pauses Renaming 44 Schools, Promises to Consult Historians
The renaming of 44 schools in the San Francisco Unified School District is apparently being put on hold after intense blowback.
Greg Keraghosian
1 min read
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.<br/><br/>
Jeff Chiu/AP
School & District Management Superintendent Who Led During COVID-19 School Shutdowns Gets Top Honors
Michelle Reid of Washington state's Northshore district, one of the very first to close schools last March, was named National Superintendent of the Year.
3 min read
Michelle Reid, superintendent of the Northshore district in Washington
Michelle Reid, the superintendent of the Northshore district in Washington, was named National Superintendent of the Year.
courtesy of AASA, the School Superintendents Association
School & District Management Is Lunchtime the 'Weak Link' in School Reopening Plans?
It's risky when students are inside and unmasked, experts say. Here are five ways to mitigate that risk and make in-school meals safer.
11 min read
Elementary students in Brownsville, Texas, eat a socially distanced lunch in the school cafeteria. Experts say there are ways to mitigate the risk of contracting COVID-19 even when kids take their masks off to eat.
Elementary students in Brownsville, Texas, eat a socially distanced lunch in the school cafeteria. Experts say there are ways to mitigate the risk of contracting COVID-19 when kids take their masks off to eat.
Denise Cathey/The Brownsville Herald via AP
School & District Management Los Angeles Unified Shrinks Police Budget. Money Will Support Black Student Achievement
The board overseeing the Los Angeles Unified School District has cut $25 million from the budget for school police and will use the money to help fund an achievement plan for Black students.
1 min read
Demonstrators holds signs during a protest to demand the defunding of the Los Angeles school district police outside of the school board headquarters on June 23, 2020, in Los Angeles.
Demonstrators holds signs during a protest to demand the defunding of the Los Angeles school district police outside of the school board headquarters on June 23, 2020, in Los Angeles.
Marcio Jose Sanchez/AP