Recall the Great Charter School War of 2004: After the NY Times published the results of an AFT report finding that traditional public schools outperformed charters, all hell broke loose. Every charter school advocate and their mother intervened in the name of educational research, arguing that the study was fundamentally flawed and that the Times story was biased against charter schools. Shortly thereafter, charter advocates took out a full page ad in the Times blasting the study and the Times for putting it forward.
To be sure, students are not randomly assigned to charter schools, so these critiques were not without merit. So keep your eyes on the Washington Post over the next week and see if charter school advocates again swoop in to defend educational research from the bad guys who would misuse it for their own purposes. Yesterday, the Post published an almost identical analysis claiming that DC charters, which currently enroll a third of DC students, “have opened a solid academic lead over those in its traditional public schools.”
Again, for those who are willing to endure the occasional recycled rant on selection versus treatment effects:
First, that students selected into a charter lottery makes them different from those who did not. It may be that their parents are more involved in their education, that they are having a particularly bad experience at their neighborhood school, or that their parents can no longer pay for private school. Whatever the reason, families selecting in, even if they are all poor and minority kids, are different by virtue of choosing a non-neighborhood school.
Lots of choice advocates will spar on this point, and argue that everyone wants a better choice for their children, so there is no selection problem. While rhetorically effective, anyone arguing that families that choose into a charter school are the same as those who don’t is simply wrong. Saying that two-thirds of the kids are poor and that the overwhelming majority are African-American and Hispanic doesn’t solve this problem. (Btw, in the small schools context, this is Joel Klein’s favorite pasttime.) Even if charter kids had prior test scores identical to their neighborhood school peers, we still couldn’t compare charter and neighborhood school kids who didn’t opt in with any confidence because there is selection on unobservables - things like motivation and aspirations that are not measured by administrative datasets used to make these comparisons.
The only defensible approach here is to compare students who entered the charter lottery and won with those who entered the lottery and lost. But if the Post wants to extend its logic, here’s a free story idea from your friendly neighborhood Secret Santa: go ahead and compare the NAEP results of charters and public schools nationally (and for extra credit, go one step further and compare the outcomes of free lunch kids in charter and traditional public schools), and let’s get the Charter War of 2008 started.
The opinions expressed in eduwonkette are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.