Students using vouchers to attend established private schools in Cleveland are slightly outperforming their public school counterparts in language skills and science, and doing about the same in reading, math, and social studies, according to the latest independent evaluation of the program.
Perhaps the most surprising conclusion of the report by a team of Indiana University researchers, however, is that students attending private schools that sprang up specifically to serve the voucher program are performing worse in all subjects tested than both Cleveland public school students and voucher students in the other private schools.
“These students’ 4th grade performance is significantly and dramatically lower than both public school and other scholarship students,’” says the research report, principally written by Kim K. Metcalf, the director of the center for evaluation at Indiana University’s school of education in Bloomington. The findings were released last week.
The report’s mixed results for the 3-year-old Cleveland voucher program are sure to add ammunition to the long-running battle in the education research community over the evaluation of school choice initiatives.
Academic questions about the effectiveness of vouchers have gained new importance in light of the U.S. Supreme Court’s refusal last month to review a state supreme court decision that upheld the constitutionality of a similar publicly funded voucher program in Milwaukee.
Voucher advocates interpreted the justices’ action as an opening to continue with voucher experiments that include religious schools.
The Cleveland program provides vouchers worth as much as $2,250 to some 4,000 low-income students this year. A state appeals court has ruled that the inclusion of religious schools violates the state and federal constitutions. But the program has been allowed to continue pending an appeal before the Ohio Supreme Court.
Controlling Variables
The new evaluation, the second on the Cleveland program by the Indiana University team, reports on testing done last spring of 66 returning voucher students in established private schools, 28 voucher students in two new private schools, and 343 students in the Cleveland public schools.
Using CTB -McGraw Hill’s TerraNova Survey Level 13, the researchers tested the students in five subjects. Mr. Metcalf and his team of five graduate students then did two analyses of the achievement results.
The first analysis controlled for student background, demographics, and pre-voucher-program test scores, and is the same kind of analysis performed in the team’s first-year evaluation.
Under this analysis, voucher students in the private schools that predated the program scored higher than a public school control group in science and language skills, but about the same in reading, mathematics, and social studies.
The second analysis controlled for an additional three variables for which information was gathered last spring: class size, teacher education level, and teacher experience. The study found that class sizes in the already established private schools were smaller than in the public schools. However, the public schools’ teachers were more experienced and were more likely to have completed coursework beyond a bachelor’s degree.
In the second analysis, voucher students in the established private schools outperformed the public school students in language skills but did not do significantly better in science, reading, math, or social studies.
“After two years in the program, the effects of participation for these [voucher] students seem to be positive, though the magnitude of the effect is small,” the report concludes.
‘Not Defensible’
But the report paints a much different picture for students attending the Hope Academies scholarship schools, which were started by Ohio industrialist David Brennan to serve children in the voucher program.
The students in the new private schools significantly underperformed the other voucher students and the public school control group in all five subjects, the report says.
It suggests one possible reason was that voucher students entering existing private schools could rely on their peers to adjust to new surroundings, while both students and teachers in the newly established schools were thrown into a new environment.
John Morris, the chief operating officer of the Hope Academies, took issue with the report. Students in his schools were required to undergo the Indiana University assessments last spring shortly after they had completed a battery of other tests, he said.
“The kids were overtested,” he said. “Common sense tells you that you don’t test one week after another test. I would submit that those scores are not defensible.”
Mr. Metcalf’s first-year report, which found no academic advantage for voucher students, came under heavy attack from another team of school choice researchers, including Paul E. Peterson of Harvard University. Mr. Peterson re-evaluated the Cleveland data and concluded that students in voucher schools were outperforming public school students.
After reading the new Indiana University report last week, Mr. Peterson said he remained troubled about whether the data were valid.
“The problem with this study was and remains that you don’t know much about these students at baseline,” he said. “We just don’t know whether these results are a function of the schools they are attending or the characteristics of the students.”
Mr. Metcalf responded that he would place his data up for any independent review. “I would encourage anyone to do that,” he said.
