The independent evaluation by a research team from Indiana University found that 4th graders using vouchers to attend established private schools in Cleveland are slightly outperforming their public school counterparts in two subject areas and doing about the same in three others. But the data also revealed an interesting twist: Students attending private schools that sprang up specifically to serve the voucher program are performing worse in all subjects than both the public school students and the other voucher students.
“These students’ 4th grade performance is significantly and dramatically lower than both public school and other scholarship students’,” says a report on the study written by Kim Metcalf, director of the center for evaluation at Indiana University’s school of education in Bloomington.
Questions about the effectiveness of vouchers have gained new importance in light of the U.S. Supreme Court’s refusal in November to review a state court decision that upheld the constitutionality of the Milwaukee voucher program, the nation’s only other publicly funded initiative. Voucher advocates interpreted the justices’ action as an opening to continue with voucher experiments that include both private and religious schools, and they have introduced new legislation in a number of states.
This school year, the Cleveland program is providing vouchers worth as much as $2,250 to roughly 4,000 low-income students. A state appeals court has ruled that the inclusion of religious schools violates the state and federal constitutions, but the program has been allowed to continue, pending an appeal before the Ohio Supreme Court.
The new study, the second on the Cleveland program by the Indiana University team, reports on testing done last spring of 66 returning voucher students in established private schools, 28 voucher students in two new private schools, and 343 students in the Cleveland public schools. Metcalf and his team of five graduate students tested the students in five subjects and analyzed the achievement results in two ways.
The first analysis controlled for student background, demographics, and pre-voucher test scores. Under this analysis, voucher students in the private schools that predate the program scored higher in science and language skills than a public school control group but about the same in reading, mathematics, and social studies.
The second analysis controlled for an additional three variables: class size, teacher education level, and teacher experience. The study found that class sizes in the already established private schools were smaller than in the public schools. However, the public schools’ teachers were more experienced and more likely to have completed coursework beyond a bachelor’s degree. When the researchers took these factors into account, voucher students in the established private schools outperformed the public school students in language skills but did not do significantly better in science, reading, math, or social studies.
“After two years in the program, the effects of participation for these [voucher] students seem to be positive, though the magnitude of the effect is small,” the report concludes.
The findings paint a much different picture for students attending the Hope Academies, which were started by Ohio industrialist David Brennan to serve children in the voucher program. The students in the Hope schools significantly underperformed the other voucher students and the public school control group in all five subjects tested. One possible reason, according to the researchers: Unlike voucher students entering existing private schools, students in the newly established Hope schools had no peers to help them adjust.
John Morris, chief operating officer of the Hope Academies, took issue with the findings. Students in his schools, he complained, were required to undergo the Indiana University assessments shortly after they had completed a battery of other tests. “The kids were overtested,” he said. “Common sense tells you that you don’t test one week after another test. I would submit that those scores are not defensible.”
Metcalf’s first-year report on the Cleveland program, which found no academic advantage for voucher students, came under heavy attack from other school-choice researchers, including Paul Peterson of Harvard University. Peterson re-evaluated the Cleveland data and concluded that students in voucher schools were outperforming public school students. [“A Studied Opinion,” October 1998.]
After the release of the new report, Peterson questioned the validity of its data, as well. “The problem with this study was and remains that you don’t know much about these students at baseline,” he said. “We just don’t know whether these results are a function of the schools they are attending or the characteristics of the students.”
Metcalf responded that he would place his data up for any independent review. “I would encourage anyone to do that,” he said.
--Mark Walsh
