Students’ gains in mathematics after three years in a charter school run by the Knowledge Is Power Program, or KIPP, are large enough in about half of schools to significantly narrow race- and income-based achievement gaps among students, according to a study of 22 KIPP middle schools nationwide.
The report, released last month, is the first of a series focusing on the nationally known charter school network. Conducted by Mathematica Policy Research of Princeton, N.J., the study was planned and commissioned by KIPP with grants from several private foundations. Future reports will expand the sample to more KIPP charter schools, use randomized experimental research in a subset of schools, and look at measures other than state test scores.
“The consistency of the effects across most of the 22 schools and the magnitude of the effects are pretty striking and impressive,” said Brian P. Gill, a senior social scientist for Mathematica and a study author.
At about half the KIPP schools, the study found that the gains in math for students after three years in the schools were equivalent of 1.2 years of extra instruction, and .9 years of additional instruction in reading, Mr. Gill said.
‘Creaming’ Not Found
Critics of charter schools often contend the schools “cream” higher-achieving students from regular public schools, but the study didn’t find any evidence that KIPP is systematically enrolling more high-performers from their school districts. On average, the report says, KIPP middle schools have students who are more likely to be living in poverty and are more likely to be black or Hispanic than are students from the schools around them. Back when they were in 4th grade, the study also found, a majority of the KIPP middle school students had lower average test scores than did students in their local school districts.
The study also looked at student attrition rates, which at least one previous study found to be higher in KIPP schools. The Mathematica researchers found, however, that while attrition rates vary widely among the schools, KIPP middle schools don’t have systematically higher or lower numbers of students leaving before completion than other schools within their districts.
“This report provides irrefutable evidence that KIPP is not getting [positive] results because of getting more able students in the door or losing students at higher rates than their counterparts,” said Steve Mancini, the director of public affairs for KIPP, which operates a total of 82 schools across the country serving 21,000 students.
But the study did show that KIPP middle schools are much less likely to enroll special education students and students with limited English skills than the school districts from which they draw.
The study also shows that students are more likely to repeat a grade in KIPP schools than in regular public schools in their districts, particularly in the 5th and 6th grades. The authors write that the differences in retention rates “likely capture KIPP’s philosophy that students should be promoted to the next grade level only after they’ve demonstrated mastery of their current grade’s material.”
Gaps Closing
The study is the first report that applies a “rigorous (nonexperimental) methodological approach” across a national sample of KIPP schools, it says. It compares the educational progress of KIPP students on state reading and math exams over four years to that of demographically similar students from the same districts.
Half of KIPP schools studied are producing achievement impacts estimated to be big enough to reduce the gap in black-white test scores in math in half within three years, the report says. Three-year effects for reading are also large, but not as large as the impact reported for math.
Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, said the methodology of the study is sound and the math gains are “impressive.”
But he said the reading gains are “modest and perhaps overstated.” They may not be as large as indicated by the researchers, he said, because when KIPP students were matched with similar students in their local school districts, the researchers only looked at some characteristics linked to students’ success. For example, students were matched according to their starting achievement level and whether or not they were special education students.
The study didn’t consider whether a parent sits with his or her child to do homework or takes the child to the public library, Mr. Fuller said.
But Mr. Gill said that he believes the matching methodology holds up because it includes the academic achievement for grades prior to enrolling in KIPP middle schools. “The only way that the results we’re seeing would be biased is if the parental effect were larger in grades 5 to 8 [the grades of students in the study] than in grades 3 or 4,” he said.
Robin J. Lake, the associate director for the Center on Reinventing Public Education, a nonpartisan research group in Seattle, said she wondered if the 22 schools studied are representative of all KIPP schools or whether some were older and more stable.
Mr. Gill said that researchers tried to get data for all 35 KIPP schools that were opened by 2005, but could only obtain data for 22 of them.
