School Choice & Charters

Vouchers Yield Mixed Results, Report Says

By Mark Walsh — November 25, 1998 4 min read
  • Save to favorites
  • Print

Students using vouchers to attend established private schools in Cleveland are slightly outperforming their public school counterparts in language skills and science, and doing about the same in reading, math, and social studies, according to the latest independent evaluation of the program.

Perhaps the most surprising conclusion of the report by a team of Indiana University researchers, however, is that students attending private schools that sprang up specifically to serve the voucher program are performing worse in all subjects tested than both Cleveland public school students and voucher students in the other private schools.

“These students’ 4th grade performance is significantly and dramatically lower than both public school and other scholarship students,’” says the research report, principally written by Kim K. Metcalf, the director of the center for evaluation at Indiana University’s school of education in Bloomington. The findings were released last week.

The report’s mixed results for the 3-year-old Cleveland voucher program are sure to add ammunition to the long-running battle in the education research community over the evaluation of school choice initiatives.

Academic questions about the effectiveness of vouchers have gained new importance in light of the U.S. Supreme Court’s refusal last month to review a state supreme court decision that upheld the constitutionality of a similar publicly funded voucher program in Milwaukee.

Voucher advocates interpreted the justices’ action as an opening to continue with voucher experiments that include religious schools.

The Cleveland program provides vouchers worth as much as $2,250 to some 4,000 low-income students this year. A state appeals court has ruled that the inclusion of religious schools violates the state and federal constitutions. But the program has been allowed to continue pending an appeal before the Ohio Supreme Court.

Controlling Variables

The new evaluation, the second on the Cleveland program by the Indiana University team, reports on testing done last spring of 66 returning voucher students in established private schools, 28 voucher students in two new private schools, and 343 students in the Cleveland public schools.

Using CTB -McGraw Hill’s TerraNova Survey Level 13, the researchers tested the students in five subjects. Mr. Metcalf and his team of five graduate students then did two analyses of the achievement results.

The first analysis controlled for student background, demographics, and pre-voucher-program test scores, and is the same kind of analysis performed in the team’s first-year evaluation.

Under this analysis, voucher students in the private schools that predated the program scored higher than a public school control group in science and language skills, but about the same in reading, mathematics, and social studies.

The second analysis controlled for an additional three variables for which information was gathered last spring: class size, teacher education level, and teacher experience. The study found that class sizes in the already established private schools were smaller than in the public schools. However, the public schools’ teachers were more experienced and were more likely to have completed coursework beyond a bachelor’s degree.

In the second analysis, voucher students in the established private schools outperformed the public school students in language skills but did not do significantly better in science, reading, math, or social studies.

“After two years in the program, the effects of participation for these [voucher] students seem to be positive, though the magnitude of the effect is small,” the report concludes.

‘Not Defensible’

But the report paints a much different picture for students attending the Hope Academies scholarship schools, which were started by Ohio industrialist David Brennan to serve children in the voucher program.

The students in the new private schools significantly underperformed the other voucher students and the public school control group in all five subjects, the report says.

It suggests one possible reason was that voucher students entering existing private schools could rely on their peers to adjust to new surroundings, while both students and teachers in the newly established schools were thrown into a new environment.

John Morris, the chief operating officer of the Hope Academies, took issue with the report. Students in his schools were required to undergo the Indiana University assessments last spring shortly after they had completed a battery of other tests, he said.

“The kids were overtested,” he said. “Common sense tells you that you don’t test one week after another test. I would submit that those scores are not defensible.”

Mr. Metcalf’s first-year report, which found no academic advantage for voucher students, came under heavy attack from another team of school choice researchers, including Paul E. Peterson of Harvard University. Mr. Peterson re-evaluated the Cleveland data and concluded that students in voucher schools were outperforming public school students.

After reading the new Indiana University report last week, Mr. Peterson said he remained troubled about whether the data were valid.

“The problem with this study was and remains that you don’t know much about these students at baseline,” he said. “We just don’t know whether these results are a function of the schools they are attending or the characteristics of the students.”

Mr. Metcalf responded that he would place his data up for any independent review. “I would encourage anyone to do that,” he said.

Related Tags:

A version of this article appeared in the December 02, 1998 edition of Education Week as Vouchers Yield Mixed Results, Report Says

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School Choice & Charters Tracker Which States Have Private School Choice?
Education savings accounts, voucher, and tax-credit scholarships are growing. This tracker keeps tabs on them so you don't have to.
School Choice & Charters Opinion What's the State of Charter Schools Today?
Even though there's momentum behind the charter school movement, charters face many of the same challenges as traditional public schools.
10 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School Choice & Charters As Private School Choice Grows, Critics Push for More Guardrails
Calls are growing for more scrutiny over where state funds for private school choice go and how students are faring in the classroom.
7 min read
Illustration of completed tasks, accomplishment, finished checklist, achievement or project progression concept. Person holding pencil tick all completed task checkbox.
Nuthawut Somsuk/iStock/Getty
School Choice & Charters How a District Hopes to Save an ESSER-Funded Program
As a one-time infusion of federal funding expires, districts are searching for creative ways to keep programs they funded with it running.
6 min read
Chicago charter school teacher Angela McByrd works on her laptop to teach remotely from her home in Chicago, Sept. 24, 2020.
Chicago charter school teacher Angela McByrd works on her laptop to teach remotely from her home in Chicago, Sept. 24, 2020. In Montana, a district hopes to save a virtual instruction program by converting it into a charter school.
Nam Y. Huh/AP