School Choice & Charters

Vouchers Yield Mixed Results, Report Says

By Mark Walsh — November 25, 1998 4 min read

Students using vouchers to attend established private schools in Cleveland are slightly outperforming their public school counterparts in language skills and science, and doing about the same in reading, math, and social studies, according to the latest independent evaluation of the program.

Perhaps the most surprising conclusion of the report by a team of Indiana University researchers, however, is that students attending private schools that sprang up specifically to serve the voucher program are performing worse in all subjects tested than both Cleveland public school students and voucher students in the other private schools.

“These students’ 4th grade performance is significantly and dramatically lower than both public school and other scholarship students,’” says the research report, principally written by Kim K. Metcalf, the director of the center for evaluation at Indiana University’s school of education in Bloomington. The findings were released last week.

The report’s mixed results for the 3-year-old Cleveland voucher program are sure to add ammunition to the long-running battle in the education research community over the evaluation of school choice initiatives.

Academic questions about the effectiveness of vouchers have gained new importance in light of the U.S. Supreme Court’s refusal last month to review a state supreme court decision that upheld the constitutionality of a similar publicly funded voucher program in Milwaukee.

Voucher advocates interpreted the justices’ action as an opening to continue with voucher experiments that include religious schools.

The Cleveland program provides vouchers worth as much as $2,250 to some 4,000 low-income students this year. A state appeals court has ruled that the inclusion of religious schools violates the state and federal constitutions. But the program has been allowed to continue pending an appeal before the Ohio Supreme Court.

Controlling Variables

The new evaluation, the second on the Cleveland program by the Indiana University team, reports on testing done last spring of 66 returning voucher students in established private schools, 28 voucher students in two new private schools, and 343 students in the Cleveland public schools.

Using CTB -McGraw Hill’s TerraNova Survey Level 13, the researchers tested the students in five subjects. Mr. Metcalf and his team of five graduate students then did two analyses of the achievement results.

The first analysis controlled for student background, demographics, and pre-voucher-program test scores, and is the same kind of analysis performed in the team’s first-year evaluation.

Under this analysis, voucher students in the private schools that predated the program scored higher than a public school control group in science and language skills, but about the same in reading, mathematics, and social studies.

The second analysis controlled for an additional three variables for which information was gathered last spring: class size, teacher education level, and teacher experience. The study found that class sizes in the already established private schools were smaller than in the public schools. However, the public schools’ teachers were more experienced and were more likely to have completed coursework beyond a bachelor’s degree.

In the second analysis, voucher students in the established private schools outperformed the public school students in language skills but did not do significantly better in science, reading, math, or social studies.

“After two years in the program, the effects of participation for these [voucher] students seem to be positive, though the magnitude of the effect is small,” the report concludes.

‘Not Defensible’

But the report paints a much different picture for students attending the Hope Academies scholarship schools, which were started by Ohio industrialist David Brennan to serve children in the voucher program.

The students in the new private schools significantly underperformed the other voucher students and the public school control group in all five subjects, the report says.

It suggests one possible reason was that voucher students entering existing private schools could rely on their peers to adjust to new surroundings, while both students and teachers in the newly established schools were thrown into a new environment.

John Morris, the chief operating officer of the Hope Academies, took issue with the report. Students in his schools were required to undergo the Indiana University assessments last spring shortly after they had completed a battery of other tests, he said.

“The kids were overtested,” he said. “Common sense tells you that you don’t test one week after another test. I would submit that those scores are not defensible.”

Mr. Metcalf’s first-year report, which found no academic advantage for voucher students, came under heavy attack from another team of school choice researchers, including Paul E. Peterson of Harvard University. Mr. Peterson re-evaluated the Cleveland data and concluded that students in voucher schools were outperforming public school students.

After reading the new Indiana University report last week, Mr. Peterson said he remained troubled about whether the data were valid.

“The problem with this study was and remains that you don’t know much about these students at baseline,” he said. “We just don’t know whether these results are a function of the schools they are attending or the characteristics of the students.”

Mr. Metcalf responded that he would place his data up for any independent review. “I would encourage anyone to do that,” he said.

Related Tags:

A version of this article appeared in the December 02, 1998 edition of Education Week as Vouchers Yield Mixed Results, Report Says


School & District Management Live Event EdWeek Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School Choice & Charters Oklahoma Charter Schools Granted Local Tax Revenue in 'Seismic' Settlement
A groundbreaking settlement will fundamentally change the way charter schools are funded in Oklahoma, despite vehement opposition.
Nuria Martinez-Keel, The Oklahoman
3 min read
This July 19, 2019 photo shows an Epic Charter Schools office in Oklahoma City. The Oklahoma State Board of Education voted Thursday in favor of an agreement with the state's public charter school association to settle a 2017 lawsuit.
This July 19, 2019 photo shows an Epic Charter Schools office in Oklahoma City. The Oklahoma State Board of Education voted Thursday in favor of an agreement with the state's public charter school association to settle a 2017 lawsuit.
Sue Ogrocki/AP
School Choice & Charters COVID-19 May Energize Push for School Choice in States. Where That Leads Is Unclear
The pandemic is driving legislators' interest in mechanisms like education savings accounts, but the growth may not be straightforward.
8 min read
Iowa Gov. Kim Reynolds delivers her Condition of the State address before a joint session of the Iowa Legislature on Jan. 12 at the statehouse in Des Moines, Iowa.
Iowa Gov. Kim Reynolds delivers her Condition of the State address to state lawmakers on Jan. 12. She's pushing a major school choice expansion.
Bryon Houlgrave/The Des Moines Register via AP
School Choice & Charters Letter to the Editor Are NOLA Charters a Mixed Bag?
To the Editor:
The opinion essay by Douglas N. Harris about how New Orleans’ education reforms post-Katrina are relevant to the COVID-19 era (“As Schools Recover After COVID-19, Look to New Orleans,” Sept. 30, 2020) highlights some basic improvements in the NOLA system but downplays the most significant aspects of those changes: the impact on people of color.
1 min read
School Choice & Charters Home Schooling Is Way Up With COVID-19. Will It Last?
The shift could have lasting effects on both public schools and the home-schooling movement.