Education Chat

Charter School Achievement: What We Know

Charter school expert Bryan C. Hassel, took questions about charter schools, what the research says about how well they are doing, and the challenges for improving and expanding the model.

August 15, 2007

Charter School Achievement: What We Know

Guest:
Bryan C. Hassel
, a co-director of Public Impact, a Chapel Hill, N.C., education policy and consulting firm. He is a co-author of Charter School Achievement: What We Know, a review of scholarly studies comparing charter vs. district school achievement for the National Alliance of Public Charter Schools. Along with numerous other articles and monographs, he is author of The Charter School Challenge, published by the Brookings Institution (1998) and co-author of Picky Parent Guide: Choose Your Child’s School with Confidence (The Elementary Years K-6), published by Armchair Press (2004).

Kathleen Kennedy Manzo (Moderator):

Welcome to our live chat on the value and effectiveness of charter schools. Thank you for joining us. Our guest, charter school expert Bryan C. Hassel of Public Impact, is awaiting your questions. I am Kathleen Kennedy Manzo, an associate editor at Education Week, and I will be your moderator. Let’s begin.

Question from Kathleen Kennedy Manzo:

Can you briefly describe your research on charter schools and your general views on the charter school movement.

Bryan C. Hassel:

I and my organization, Public Impact, have conducted numerous studies of charter school issues ranging from finance to authorizing to state policy. While we have not carried out any large-scale quantitative studies of charter student achievement, I have had the chance to review most such studies in the process of summarizing them for the National Alliance for Public Charter Schools paper “Charter School Achievement: What Do We Know?” (http://www.publicimpact.com/high-stakes/CS%20Achievement%20Studies%20Oct06%20Update.pdf)

As for my views on charter schools, I would say I am an advocate for giving the charter idea a full test while working to improve the outcomes achieved from chartering over time. This means I support charter school laws that allow a good number of schools to open and give them autonomy and fair funding, but also that I believe strongly in rigorous up-front authorizing and tough accountability, including closing chronic low-performers.

Question from Dr. Jo Campbell, Professional Educator:

Do you have any data to share comparing value added charter school student achievement compared to their local public schools?

Bryan C. Hassel:

While there have been dozens of studies purporting to answer this question, few are sufficiently rigorous or have access to the necessary data to provide a valid answer.*

Recently, some scholars have begun to employ the “gold standard” method of comparing charter school students with those who were randomly lotteried out of charter schools. Since these studies compare charter and non-charter students who are otherwise similar, they arguably produce the best estimates of the value added effects of charter schools. The largest and most recent of these is Hoxby and Murarka’s study of NYC charter schools, which found that charter students gained modestly more in math and reading than non-charter students -- 0.09 and 0.04 standard deviations per year. These are modest gains but would accumulate over time.(http://www.nber.org/~schools/charterschoolseval/)

Without random lotteries, other researchers have looked at large-scale datasets and tried to use statistical methods to control for differences in charter and non-charter students. These studies have been very mixed in their results, with negative results for charter schools in states such NC and positive results in other states such as DE.

There is simply no way to glean from all of these studies some kind of answer to the question “are charter schools adding more value?” The results vary widely. Our review of studies for the National Alliance for Public Charter Schools found that studies that chart change over time tend to find higher levels of improvement in charter schools. These averages, though, mask vast diversity within the sector. There are many fabulous charter schools, struggling charter schools, and schools in the middle.

* For those interested in the methodological issues, I’d suggest a couple of papers on the topic, one from a “consensus panel” convened by the National Charter School Research Project (http://www.ncsrp.org/cs/csr/print/csr_docs/pubs/panel.htm) and one by researchers Hoxby and Murarka ( draft at http://www.vanderbilt.edu/schoolchoice/conference/papers/Hoxby-Murarka_2006-DRAFT.pdf)

Question from Jim Whiteman, Center Faculty, The Center for Leadership in Education, Elyria Ohio:

Among what appears to me mixed results in testing, WHY are so many parents (and teachers) choosing charter schools and what does that tell us?

Bryan C. Hassel:

Let me focus on an implied version of this question, which is: why do so many parents seem willing to choose charter schools with low test scores? Parents may value other aspects of charter schools, such as their small size, communal culture, or safety -- all things that come up high in surveys of charter parents. In addition, they may find that despite charters’ low scores, the scores in their alternatives are even lower, at least for children like theirs. Finally, parents may simply be mis-informed about relative school results in some cases.

Kathleen Kennedy Manzo (Moderator):

You can view Ed Week’s coverage of charter schools, including the story on the NYC study, in our archives and from our Research Center...Here are a couple of direct links.

http://www.edweek.org/ew/articles/2007/08/01/44charter.h26.html

http://www.edweek.org/rc/issues/charter-schools/

Question from Clyde Barrow:

The Rand study of managed charter schools in Philadelphia, showed them performing at the same or even lower levels as the neighborhood schools they were supposed to replace--and at nearly twice the cost. Have you found special problems with EMO-run charter schools?

Bryan C. Hassel:

This question points to one feature of this whole topic that needs highlighting -- within the sector of “charter schools,” there are many many different types of schools. This question points to one difference -- schools managed by EMOs or charter management organziations vs. stand-alone schools. Generally, studies have not found an advantage one way or the other. But within the category of EMO/CMO-managed schools, of course, there is further diversity. Some networks, like KIPP, have achieved extraordinary results while others have lagged.

Beyond that difference, there’s wide diversity within the charter sector related to curriculum, instruction, school culture, governance, and management. In that context, it’s questionable how relevant it is to compare “average” charter achievement levels with average district levels. WHat’s of more interest is what kinds of schools are being formed with chartering, and then how different kinds of schools are doing. The overall horse-race is, frankly, not all that interesting or important -- yet it soaks up much media and political attention.

One clarifying note on Philly specifically: I don’t think many of the EMO-managed schools are actually charter schools; they operate under some other kind of contract with the district.

Question from Giulia Cox, Executive Director, Student Support Services NYC DOE Dist 79:

Do you see a regional pattern in terms of charter schools’ best practices or underperformance? Are there differences, for example, among urban/rural, west/east, college town / non-college town implementations?

Bryan C. Hassel:

I’m not aware of any patterns like this that seem to hold across the country. Generally, there’s a great deal of diversity within all these regional categories that’s more predictive of practices and performance than these regional factors are likely to be.

Question from Cathy O’connell, Teacher, Hillside School, Hillside, AZ:

Do you believe Charter Schools fill a void in public Education for smaller, child centered schools regardless of whether they “do better” than tradidional public schools?

Bryan C. Hassel:

I absolutely think charter schools provide various things that parents value, regardless of whether they “do better” than traditional public schools on test measures, and one of those values is what the questioner mentions.

That said, there’s an important policy question around what should count as “valuable” when it comes to public accountability. One extreme of this debate says that if parents value it, then the public should value it; i.e, let the market decide what’s important. If schools provide a small learner centered environment that parents like, great -- even if the kids don’t learn to read and do math.

I don’t take that view, however. It seems to me reasonable, even imperative, to insist that public schools achieve a baseline of publicly valued results as a condition of continuing to be public schools. At the elementary and middle level, this is easier to define, b/c most people would agree that schools at this level need to be teaching kids the 3 R’s. High school is tougher, because we have less consensus about what the publicly valued results are. But I’d still stick by having some kind of results-based measure. If parents value other things and want to make choices on those bases, great -- but public schools should all also deliver the basic results we agree on.

Question from Kym Rich, Parent, Kennesaw Charter School:

How well are the charter schools doing in regards to special needs children, specifically those with ADHD and other behavioral issues?

Bryan C. Hassel:

For folks interested in the range of issues related to special ed and charters, I’d point to the website for Project Intersect, which was a federally funded multi-year effort to study this. (http://www.education.umd.edu/EDSP/ProjectIntersect/index.html)

This project did a nice job of examining all the tough issue that arise in the context of charter schools providing special ed, and explores how different schools and states and charter authorizers have dealt with them.

One of the products of this work looks at achievement outcomes of students with disabilities in California. Its finding on outcomes was, like a lot of charter achievement studies, encouraging but very tentative: “The population of students studied also posted similar or better outcomes than their peers in traditional public schools did, but important questions remain regarding inferences that may be drawn from this finding.”

Question from Keith MacAllum, PhD, Senior Study Director, Westat:

Evaluative research on certain charter school models has yielded impressive results. Strong community and private sector involvement helps account for their success. What factors promote and inhibit replication of proven models in other communities?

Bryan C. Hassel:

This is one of the key question facing the charter world. Performance is mixed, with some outstanding schools, and so the natural question is how to “get more of those” through replication elsewhere (as well as new start-ups that form the next generation of stand-outs).

As for the question of what factors promote and inhibit replication elsewhere, I’d start by pointing out that there’s little evidence that this is likely to happen naturally, through imitation. There are some great charter schools, but there have been great public schools long before charters entered the scene, and yet you simply don’t see the rush to imitate, for a whole host of factors.

So if we’re going to see wide-scale replication, it’s going to be because successful schools form “charter management organizations” or other networks to take their successes to scale, or because third parties emerge that specialize in replication at scale. These entities need access to capital and talent, and so those are two key factors. Another is whether the policy environment is hospitable to replication. Charter laws that require each school to have a separate board, for example, may get in the way of networked scale up.

Question from Ben Sherman, Principal, East-West School, NYC:

Can you compare the teacher turnover rate at charter schools to that of public schools. It seems like the best of the charter schools push the teachers so hard that many leave at the end of the year, forcing the school to continually reinvent the wheel. Is this perception or truth?

Bryan C. Hassel:

There is some evidence that charter teacher turnover rates are higher. One theory is the one you pose -- that they’re pushed hard and burn out, especially in the start-up years. Other forces are at work as well, though. One,many charter schools seek to build a unique culture and use a specific set of approaches, which are different from the norm. Some teachers will like this approach and others won’t, and so there’s a natural sorting process.

Two, charter school leadership tends to have much more authority to replace teachers who don’t perform, don’t fit the culture, etc. So there’s likely to be much more “involuntary turnover” in charter schools, pardon my euphemism!

A key issueis whether the resulting turnover is “bad” or “good.” The questioner focuses on the need to “reinvent the wheel,” but on the plus side over time a charter faculty is likely to coalesce more and more around a common approach and mission, which is good. Turnover rates in conventional public schools are actually quite low, compared to other professional services occupations, according to Labor dept data.

Question from Jill Levy , President, American Federation of School Administrators:

Public school administrators across the nation experience a flow of students who are rejected by their charter schools to return to their local public schools and adversely affect school statistics. What steps are taken to hold charter schools to the same suspension and rejection standards as their public shool counterparts?

Bryan C. Hassel:

It’s interesting how the accusations fly both ways on this question, without a lot of data to back up either side. Charter administrators often complain to me, for example, that the local district encourages its most troublesome kids (and parents!) to apply to the charter school. That said, the question raises a real dilemma for charter policy. On one hand, part of having autonomy as a school means defining expectations for behavior and holding to those standards. On the other, if charter schools can send away students routinely and districts can’t, it’s pretty tough to make apples-to-apples comparisons on performance. Ultimately, I come down on the side of thinking charter authorizers need to do more to make sure charter schools are educating all-comers, and finding alternatives to expulsion when problems arise. And the same goes for district schools.

Kathleen Kennedy Manzo (Moderator):

There is some information from the U.S. Department of Education that might be useful to some of you.. and they have a link to a listserv on the topic at http://www.ed.gov/programs/charter/index.html

Another link here: http://www.uscharterschools.org/pub/uscs_docs/index.htm

Question from Ann Otwell, Middle School Graduation coach, Chestnut Log Middle School, Douglas county, Georgia:

As a Middle School Graduation Coach I work with at-risk students, helping them to find their place and increase their chances of graduation from high school. Are charter schools doing a better job of improving graduation rates? Why or why not?

Bryan C. Hassel:

There’s mostly anecdotal evidence on this point, with stories of specific charter schools that achieve extraordinary graduation rates with populations who typically experience a lot of drop outs. I don’t know of large-scale studies that compare charter v. non-charter more broadly on this point.

DC’s SEED school and Boston’s MATCH are two that come to mind, but there many other cases. The story in these schools is often quite the same: (1) a thoroughgoing expectation that everybody’s going to graduate (2) relentless intervention with kids who appear at risk of dropping out, and (3) college counseling and assistance of the sort we more typically associate with elite high schools. This is not to say these are norms across all charter schools by any means, just in these standouts.

Question from John Smith, Principal, Greenvale Elementary:

I have a question about the larger political impact of charters. Even if they were to have no discernible effect--positive or negative--on student achievement, couldn’t they be justified as a way to keep disgruntled parents in the public school system?

Bryan C. Hassel:

Some people certainly think this way. One way charter laws have been passed in some states is as an alternative to the dreaded voucher plan that would allow parents to use public money in private schools. Charters look palatable by comparison to policymakers who don’t like the look of vouchers.

Personally, I don’t think this is a sufficient justification for chartering. Though performance from school to school is bound to be mixed, chartering needs to produce higher performance over time on metrics that matter to the public in order to be “worth it” in policy terms.

Question from Brian Berry, Teacher, Booker Middle School:

This teacher just took his daughter out of a public charter. The reason: the initial emphasis on great faculty and a lean, intelligent educational philosophy was soon eaten up by the pressure to raise money and build. Is this often the case?

Bryan C. Hassel:

This is a serious issue for charters nationwide, which typically do not receive facilities or facilities funding and thus must dig into operating funds to buy, build, or lease facilities. We often talk about the financial effects of this, but the questioner raises another important dimension: the degree to which the work of raising money, hustling for financing, and overseeing a search for a facility or a construction project diverts the attention of school leaders, faculty, and board members from academic work.

Better facilities funding would help with the $ side, and to some degree with the attention problem, but the attention problem would remain even if schools had $ to spend on facilities. That’s why I’m intrigued by the emergence of special purpose nonpriofits like NYC’s Civic Builders that have set out to do facilities work for charter schools -- getting the financing, finding buildings, and fixing them up -- so that school leaders can focus on getting the job done for kids.

Kathleen Kennedy Manzo (Moderator):

Shameless promotion here of more Ed Week coverage:

Ed Week will take a year-long look at New Orleans, some of which will cover the charter school program there. Here’s the intro piece by Lesli Maxwell:

City Yearns for Rebirth Among Ruin http://www.edweek.org/ew/articles/2007/08/15/45nolaoverview.h26.html

ALSO:

A recent Ed Week commentary looked at religious charter schools: What About Religious Charter Schools? http://www.edweek.org/ew/articles/2007/06/20/42cooper.h26.html

Question from George Guild, Director of Economic Education, Federal Reserv:

What mechanisms are there set up within the states and across the nation to share “promising practices” and success stories in charter schools with the public schools? What instruments exist today to leverage and implement these “promising practices”?

Bryan C. Hassel:

The truth is there are very few mechanisms like this in states or nationally. Part of it is resources and capacity -- it’s no one’s job to do this sharing and so it doesn’t get done. Part of it is politics -- in many places, charter schools and districts are in open hostility and there ain’t gonna be any sharing. And in some cases the promising things charter schools are doing aren’t possible in the conventional setting due to collective bargaining agreements, school board policies, or state regulations and statutes that don’t apply to charter schools.

But the bigger issue is that in public education in general, we don’t do a great job of adopting practices widely that appear to work in one setting. Stand-out schools (charter and non) may get featured on 60 Minutes but that doesn’t mean then that other schools everywhere clamor to implement their practices. Even well-funded efforts by, say, the NSF to spread effective practices often have fallen on infertile ground.

For all these reasons I think the best prospects for “leveraging and implementing promising practices” is the formation of new schools, charter and non, that are based on the successes of other schools. This kind of “replication” is difficult as well, but it sidesteps some of the challenges that have prevented good ideas from spreading.

Question from SGlass, PhD student, Liberty Univ.:

In your opinion what are the major differences and similarities between a charter school and a magnet school? Are the processes similar to get one established?

Bryan C. Hassel:

The primary difference is that charter schools tend to be initiated & operated by people outside of a school system, such as parents, teachers acting as individuals, and community organziations, while magnet schools tend to be initiated & operated by school districts to achieve some purpose like racial desegregation or offering a school with a certain curricular focus.

Legally, charter schools are typically independent organizations that operate under a contract with an authorizer. Magnet schools, by contrast, are legally just schools within a school district -- they have no independent status.

Question from Daniel Baron, Principal Teacher, The Project School, Bloominton, IN:

How can we target historically disenfranchised populations in an open lottery? I am concerned that we will attract an affluent student body and our school’s vision is to eliminate the predictive value of race, class, language, gender, sexual orientation and special abilities on success in school and in life.

Bryan C. Hassel:

Different states have different rules related to lotteries, and so it’s worth checking what’s possible in a given state before assuming you can’t do any kind of targeting like this. There are also federal regulations to consider if you’re receiving federal dollars. But it may well be possible to give preferences to certain neighborhoods, to kids who attend currently failing schools, and the like -- all of which could be used to get the kind of diversity you want.

It’s also worth noting that the problem can run the other way. Some schools have set out to be diverse but found that affluent parents fled (or never signed up) once the school gained a reputation as having “too many” kids from the other side of the tracks -- a classic “tipping point” phenomenon that’s also been observed in housing and other domains.

Question from Ron Newell, Director of Evaluation, EdVisions Schools:

What is being done to measure the affective elements of school culture and climate of charter schools? This appears to be the major reason parents and students choose them, yet all the measures are typical measures that do not differentiate school climates.

Bryan C. Hassel:

I don’t know of many serious efforts to measure this widely, say across an entire charter sector in a state. Certainly there are numerous instruments available to assess culture and climate that schools can buy and use and that scholars use to study this kind of thing, but that’s different from implementing such a measure on a wide scale and reporting the results for everyone to see.

Many charter school authorizers, however, send site visit teams into schools to examine, among other things, the factors you mention. The Mayor of Indianapolis’s charter program, for example, sends expert teams into schools every year, which produce reports that inform that annual accountability report the office puts out.

Question from Cindi Englefield, President, Show What You Know Publishing:

How are charter schools being held fiscally responsible? Who is running these schools and how do we monitor how they spend our tax payers money?

Bryan C. Hassel:

Fiscal monitoring varies greatly from place to place. Charter schools are typically required to submit financial reports of some kind to the state, district, or charter authorizer. They are in some places required to obtain external CPA audits of their financial operations. Many authorizers specifically examine how charter schools are managing their finances.

No monitoring system is perfect, however. And there is a tradeoff between how intensive monitoring is and how much value charter schools obtain from their autonomy. Requiring charter schools to submit all expenditures up a long chain of approvals, for example, would decrease the opportunity for mismanagement, but it would also undermine the nimbleness that now helps charter schools meet the needs of kids.

Kathleen Kennedy Manzo (Moderator):

There are many, many more questions, but unfortunately our time is up. Judging from the number and variety of questions it is a safe bet that this conversation will continue among educators, researchers, policymakers and parents well into the future. Thank you all for participating in this very informative discussion.

And thank you to Bryan Hassel for offering his time and his expertise.

The Fine Print

All questions are screened by an edweek.org editor and the guest speaker prior to posting. A question is not displayed until it is answered by the guest speaker. Due to the volume of questions received, we cannot guarantee that all questions will be answered, or answered in the order of submission. Guests and hosts may decline to answer any questions. Concise questions are strongly encouraged.

Please be sure to include your name and affiliation when posting your question.

Edweek.org’s Online Chat is an open forum where readers can participate in a give- and-take discussion with a variety of guests. Edweek.org reserves the right to condense or edit questions for clarity, but editing is kept to a minimum. Transcripts may also be reproduced in some form in our print edition. We do not correct errors in spelling, punctuation, etc. In addition, we remove statements that have the potential to be libelous or to slander someone. Please read our privacy policy and user agreement if you have questions.

—Chat Editors