To the Editor:
The Boston Foundation’s recently released study on the performance of Boston’s charter and “pilot” schools had a substantial limitation that was not made clear by its researchers (“Study Evaluates Boston’s Charter Schools,” Jan. 21, 2009.) The conclusions of the lottery analysis, which compared students who were accepted to the charter and pilot schools with those who applied but were not accepted, were based on 26 percent of the charter sample (zero of five elementary, four of 13 middle, and three of nine high schools)—those that had good records and waiting lists. In contrast, almost all the pilot schools were included at the elementary and middle school levels. It stands to reason that highly sought-after charter schools are high performers. Charters without waiting lists or records, including two since closed and one slated for closure because of underperformance, were left off the study. The small sample of the highest-performing charters was reported as if the positive results reflected all charters.
Furthermore, pilot schools serve substantially greater percentages of limited-English-proficient and low-income students, and students with moderate to severe special needs, than do charters. While the report analyses controlled for many individual demographic variables, the school-level effects created by these differences were not taken into account.
The study did find that pilot elementary schools fared well in both the lottery and observational analyses. Pilot high schools performed strongly and comparably to charter high schools in the observational analysis, although not as well as charters in the lottery analysis, a puzzling finding unexplained by the study’s authors.
The study confirmed what we have known: As reforms go to scale, we need to be vigilant about accountability for high performance from every school. While pilot elementary and high schools are doing well, underperformance is clustered at the middle school level with several struggling schools. What this study tells us is that we should applaud and learn from those pilots and charters that are doing well, and press for greater accountability for high performance from those schools that are not.
Dan French
Executive Director
Center for Collaborative Education
Boston, Mass.
