Voucher Plans’ Test Data Yield Puzzling Trends

By Debra Viadero — February 27, 2002 5 min read

The latest results from a pair of studies of school choice programs in New York City and the District of Columbia raise as many questions as they answer about what happens when students are given vouchers to help pay for their private school tuition.

According to the data, black students in New York City who took advantage of privately financed scholarships to private schools scored significantly higher after three years—as much as 9 percentile points—than their peers who had applied to the program but were turned down.

The same pattern did not hold, though, for African-American students taking part in a similar program in the nation’s capital or for Hispanic students in either city.

The New York City study, “School Choice in New York City After Three Years: An Evaluation of the School Choice Scholarship Program,” is available from the Mathematica Research Policy Inc. (Requires Adobe’s Acrobat Reader.)

“It looks like there’s a definitive effect for African-American students in New York, but we still have a lot of unanswered questions,” said Brian P. Gill, a social scientist at the RAND Corp., a think tank based in Santa Monica, Calif., who led a review of voucher studies last year but was not involved in either of the new studies. “This hasn’t given us any definitive, final answers on some of the important research questions or the important public-policy questions,” he said.

Plans Closely Watched

The new findings were released last week just as the U.S. Supreme Court took up the voucher issue in a case involving Cleveland’s 7-year-old school voucher program.

The voucher experiments in New York and Washington have been closely watched since their inception. That is in part because the scholarships in both cities were awarded to students from low-income families by means of a lottery, a procedure that allowed researchers to compare randomly chosen voucher recipients with those who did not get vouchers, but whose families had still gone to the trouble of applying for them. (“Privately Financed Vouchers Help Black Students, Two Studies Find,” Sept. 6, 2000.)

In New York, for instance, the families of 20,000 elementary school students applied in 1997 for just 1,500 scholarships, each worth $1,400 a year. Of the winners, 44 percent were African-American and 48 percent were Hispanic.

For the African-American students in New York City who used their vouchers to attend private schools, the test-score improvements showed up as early as the first year of the program. Compared with their public school counterparts, those students on average scored 5.5 percentile points higher that year; 4.2 percentile points better in the program’s second year; and 9.2 points higher in the spring of 2000, the third year of the program. The numbers are based on the students’ combined reading and mathematics scores from the Iowa Tests of Basic Skills.

“That raises important public-policy issues for why we are seeing gains for one group but not another,” said David E. Myers, the project director for the New York study. Paid for with grants from nine different foundations, the study is a joint project of Mathematica Policy Research Inc., an independent research firm based in Princeton, N.J., and the Program on Education Policy and Governance at Harvard University, which is headed by Paul E. Peterson, a Harvard professor and well-known voucher researcher.

D.C. Story

Researchers said they could not explain why the scores differed for black and Hispanic students in New York City. Although students from each group came from different public schools, Mr. Myers noted, the public schools the African-American students had attended did not seem to be markedly worse than those attended by Hispanic students.

“There’s got to be some complex set of factors that are particularly operating for African-American students,” said Mr. Peterson.

In the District of Columbia, by comparison, African-American students in private schools scored 9 percentile points higher than the public school comparison group after two years. But their achievement edge disappeared in the program’s third year.

Mr. Peterson, who helped with both studies, said black students’ test-score patterns may have been different in Washington because charter schools in that city were becoming better established, a development that may have drawn some students away from private schools and regular public schools and muddied the picture that researchers were trying to examine.

The findings for Washington, which were released separately from the Mathematica study, will be included in a forthcoming book by Mr. Peterson and William G. Howell, an assistant professor of political science at the University of Wisconsin-Madison. Slated to be published by the Brookings Institution, a Washington think tank, the book describes voucher experiments involving a total of 40,000 students in New York; Washington; Dayton, Ohio; and the Edgewood school district in San Antonio.

“In New York,” Mr. Peterson said, “we’re getting a more consistent portrait. ... We certainly have enough information, in my view, to do a larger experiment.”

Some other researchers, however, were skeptical of the New York findings. Gene V. Glass, a professor of public policy at Arizona State University in Tempe, said an alternative explanation for black students’ test-score patterns could be that those students get an “artificial” boost the first year as they move out of troubled city schools. In the second year, students begin to fail and drop out of private schools, leaving only the most able students in private schools in the third year, he suggested.

“It shows all the signs of a randomized experiment gone bad,” he said.

Parents surveyed for that study, though, most often cited cost as the deciding factor for moving their students out of the private schools.

Still, the inconsistency of the two cities’ results should not discourage policymakers from continuing to experiment with voucher initiatives, said Mr. Gill of RAND.

“This is the best evidence we’ve got about the effectiveness of vouchers for students who attend private schools,” he said. “We need a lot more research to understand why these results would be true, and if they would be generally true.”

Related Tags:

A version of this article appeared in the February 27, 2002 edition of Education Week as Voucher Plans’ Test Data Yield Puzzling Trends


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
How Schools Can Implement Safe In-Person Learning
In order for in-person schooling to resume, it will be necessary to instill a sense of confidence that it is safe to return. BD is hosting a virtual panel discussing the benefits of asymptomatic screening
Content provided by BD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
How Districts Are Centering Relationships and Systemic SEL for Back to School 21-22
As educators and leaders consider how SEL fits into their reopening and back-to-school plans, it must go beyond an SEL curriculum. SEL is part of who we are as educators and students, as well as
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Achievement Webinar
The Fall K-3 Classroom: What the data imply about composition, challenges and opportunities
The data tracking learning loss among the nation’s schoolchildren confirms that things are bad and getting worse. The data also tells another story — one with serious implications for the hoped for learning recovery initiatives
Content provided by Campaign for Grade-Level Reading

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion AP Exams Can't Be Business as Usual This Year
The College Board seems unconcerned with the collateral damage of its pandemic approach, writes an assistant superintendent of curriculum and instruction.
Pete Bavis
5 min read
Illustration of large boat in turbulent waters with other smaller boats falling into the abyss.
iStock/Getty Images Plus
Assessment Federal Lawmakers Urge Miguel Cardona to Let States Cancel Tests, Highlighting Discord
A letter from Democratic members to the new education secretary calls for an end to the "flawed" system of annual standardized exams.
3 min read
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
John Minchillo/AP
Assessment How Two Years of Pandemic Disruption Could Shake Up the Debate Over Standardized Testing
Moves to opt out of state tests and change how they're given threaten to reignite fights over high-stakes assessments.
9 min read
Image of a student at a desk.
Assessment A Plan for Standardized Test Scores During the Pandemic Has Gotten States' Attention
A testing expert says his idea would provide helpful data with key context, but said other measures about student well-being are crucial.
7 min read
HS class 1257213326