Voucher Plans’ Test Data Yield Puzzling Trends

By Debra Viadero — February 27, 2002 5 min read

The latest results from a pair of studies of school choice programs in New York City and the District of Columbia raise as many questions as they answer about what happens when students are given vouchers to help pay for their private school tuition.

According to the data, black students in New York City who took advantage of privately financed scholarships to private schools scored significantly higher after three years—as much as 9 percentile points—than their peers who had applied to the program but were turned down.

The same pattern did not hold, though, for African-American students taking part in a similar program in the nation’s capital or for Hispanic students in either city.

The New York City study, “School Choice in New York City After Three Years: An Evaluation of the School Choice Scholarship Program,” is available from the Mathematica Research Policy Inc. (Requires Adobe’s Acrobat Reader.)

“It looks like there’s a definitive effect for African-American students in New York, but we still have a lot of unanswered questions,” said Brian P. Gill, a social scientist at the RAND Corp., a think tank based in Santa Monica, Calif., who led a review of voucher studies last year but was not involved in either of the new studies. “This hasn’t given us any definitive, final answers on some of the important research questions or the important public-policy questions,” he said.

Plans Closely Watched

The new findings were released last week just as the U.S. Supreme Court took up the voucher issue in a case involving Cleveland’s 7-year-old school voucher program.

The voucher experiments in New York and Washington have been closely watched since their inception. That is in part because the scholarships in both cities were awarded to students from low-income families by means of a lottery, a procedure that allowed researchers to compare randomly chosen voucher recipients with those who did not get vouchers, but whose families had still gone to the trouble of applying for them. (“Privately Financed Vouchers Help Black Students, Two Studies Find,” Sept. 6, 2000.)

In New York, for instance, the families of 20,000 elementary school students applied in 1997 for just 1,500 scholarships, each worth $1,400 a year. Of the winners, 44 percent were African-American and 48 percent were Hispanic.

For the African-American students in New York City who used their vouchers to attend private schools, the test-score improvements showed up as early as the first year of the program. Compared with their public school counterparts, those students on average scored 5.5 percentile points higher that year; 4.2 percentile points better in the program’s second year; and 9.2 points higher in the spring of 2000, the third year of the program. The numbers are based on the students’ combined reading and mathematics scores from the Iowa Tests of Basic Skills.

“That raises important public-policy issues for why we are seeing gains for one group but not another,” said David E. Myers, the project director for the New York study. Paid for with grants from nine different foundations, the study is a joint project of Mathematica Policy Research Inc., an independent research firm based in Princeton, N.J., and the Program on Education Policy and Governance at Harvard University, which is headed by Paul E. Peterson, a Harvard professor and well-known voucher researcher.

D.C. Story

Researchers said they could not explain why the scores differed for black and Hispanic students in New York City. Although students from each group came from different public schools, Mr. Myers noted, the public schools the African-American students had attended did not seem to be markedly worse than those attended by Hispanic students.

“There’s got to be some complex set of factors that are particularly operating for African-American students,” said Mr. Peterson.

In the District of Columbia, by comparison, African-American students in private schools scored 9 percentile points higher than the public school comparison group after two years. But their achievement edge disappeared in the program’s third year.

Mr. Peterson, who helped with both studies, said black students’ test-score patterns may have been different in Washington because charter schools in that city were becoming better established, a development that may have drawn some students away from private schools and regular public schools and muddied the picture that researchers were trying to examine.

The findings for Washington, which were released separately from the Mathematica study, will be included in a forthcoming book by Mr. Peterson and William G. Howell, an assistant professor of political science at the University of Wisconsin-Madison. Slated to be published by the Brookings Institution, a Washington think tank, the book describes voucher experiments involving a total of 40,000 students in New York; Washington; Dayton, Ohio; and the Edgewood school district in San Antonio.

“In New York,” Mr. Peterson said, “we’re getting a more consistent portrait. ... We certainly have enough information, in my view, to do a larger experiment.”

Some other researchers, however, were skeptical of the New York findings. Gene V. Glass, a professor of public policy at Arizona State University in Tempe, said an alternative explanation for black students’ test-score patterns could be that those students get an “artificial” boost the first year as they move out of troubled city schools. In the second year, students begin to fail and drop out of private schools, leaving only the most able students in private schools in the third year, he suggested.

“It shows all the signs of a randomized experiment gone bad,” he said.

Parents surveyed for that study, though, most often cited cost as the deciding factor for moving their students out of the private schools.

Still, the inconsistency of the two cities’ results should not discourage policymakers from continuing to experiment with voucher initiatives, said Mr. Gill of RAND.

“This is the best evidence we’ve got about the effectiveness of vouchers for students who attend private schools,” he said. “We need a lot more research to understand why these results would be true, and if they would be generally true.”

Related Tags:

A version of this article appeared in the February 27, 2002 edition of Education Week as Voucher Plans’ Test Data Yield Puzzling Trends


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Here to Stay – Pandemic Lessons for EdTech in Future Development
What technology is needed in a post pandemic district? Learn how changes in education will impact development of new technologies.
Content provided by AWS
School & District Management Live Online Discussion A Seat at the Table: Strategies & Tips for Complex Decision-Making
Schools are working through the most disruptive period in the history of modern education, facing a pandemic, economic problems, social justice issues, and rapid technological change all at once. But even after the pandemic ends,

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
INVALSI Addresses Italy’s COVID-19 Learning Loss
Find out how INVALSI worked with TAO to develop a plan of action that can serve as a model for other education leaders grappling with the...
Content provided by TAO by Open Assessment Technologies
Assessment Biden Administration's Level of Tolerance for Cutting Standardized Tests Comes Into Focus
A distinction has grown between states having to make tests available, and districts deciding it's not practical to make students take them.
8 min read
Image of a test sheet.
Assessment Opinion Alternatives to Standardized Tests During a Pandemic Year
Three educators suggest alternatives to federally mandated standardized testing during this year undercut by COVID-19.
7 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."