Voucher Plans’ Test Data Yield Puzzling Trends

By Debra Viadero — February 27, 2002 5 min read
  • Save to favorites
  • Print

The latest results from a pair of studies of school choice programs in New York City and the District of Columbia raise as many questions as they answer about what happens when students are given vouchers to help pay for their private school tuition.

According to the data, black students in New York City who took advantage of privately financed scholarships to private schools scored significantly higher after three years—as much as 9 percentile points—than their peers who had applied to the program but were turned down.

The same pattern did not hold, though, for African-American students taking part in a similar program in the nation’s capital or for Hispanic students in either city.

The New York City study, “School Choice in New York City After Three Years: An Evaluation of the School Choice Scholarship Program,” is available from the Mathematica Research Policy Inc. (Requires Adobe’s Acrobat Reader.)

“It looks like there’s a definitive effect for African-American students in New York, but we still have a lot of unanswered questions,” said Brian P. Gill, a social scientist at the RAND Corp., a think tank based in Santa Monica, Calif., who led a review of voucher studies last year but was not involved in either of the new studies. “This hasn’t given us any definitive, final answers on some of the important research questions or the important public-policy questions,” he said.

Plans Closely Watched

The new findings were released last week just as the U.S. Supreme Court took up the voucher issue in a case involving Cleveland’s 7-year-old school voucher program.

The voucher experiments in New York and Washington have been closely watched since their inception. That is in part because the scholarships in both cities were awarded to students from low-income families by means of a lottery, a procedure that allowed researchers to compare randomly chosen voucher recipients with those who did not get vouchers, but whose families had still gone to the trouble of applying for them. (“Privately Financed Vouchers Help Black Students, Two Studies Find,” Sept. 6, 2000.)

In New York, for instance, the families of 20,000 elementary school students applied in 1997 for just 1,500 scholarships, each worth $1,400 a year. Of the winners, 44 percent were African-American and 48 percent were Hispanic.

For the African-American students in New York City who used their vouchers to attend private schools, the test-score improvements showed up as early as the first year of the program. Compared with their public school counterparts, those students on average scored 5.5 percentile points higher that year; 4.2 percentile points better in the program’s second year; and 9.2 points higher in the spring of 2000, the third year of the program. The numbers are based on the students’ combined reading and mathematics scores from the Iowa Tests of Basic Skills.

“That raises important public-policy issues for why we are seeing gains for one group but not another,” said David E. Myers, the project director for the New York study. Paid for with grants from nine different foundations, the study is a joint project of Mathematica Policy Research Inc., an independent research firm based in Princeton, N.J., and the Program on Education Policy and Governance at Harvard University, which is headed by Paul E. Peterson, a Harvard professor and well-known voucher researcher.

D.C. Story

Researchers said they could not explain why the scores differed for black and Hispanic students in New York City. Although students from each group came from different public schools, Mr. Myers noted, the public schools the African-American students had attended did not seem to be markedly worse than those attended by Hispanic students.

“There’s got to be some complex set of factors that are particularly operating for African-American students,” said Mr. Peterson.

In the District of Columbia, by comparison, African-American students in private schools scored 9 percentile points higher than the public school comparison group after two years. But their achievement edge disappeared in the program’s third year.

Mr. Peterson, who helped with both studies, said black students’ test-score patterns may have been different in Washington because charter schools in that city were becoming better established, a development that may have drawn some students away from private schools and regular public schools and muddied the picture that researchers were trying to examine.

The findings for Washington, which were released separately from the Mathematica study, will be included in a forthcoming book by Mr. Peterson and William G. Howell, an assistant professor of political science at the University of Wisconsin-Madison. Slated to be published by the Brookings Institution, a Washington think tank, the book describes voucher experiments involving a total of 40,000 students in New York; Washington; Dayton, Ohio; and the Edgewood school district in San Antonio.

“In New York,” Mr. Peterson said, “we’re getting a more consistent portrait. ... We certainly have enough information, in my view, to do a larger experiment.”

Some other researchers, however, were skeptical of the New York findings. Gene V. Glass, a professor of public policy at Arizona State University in Tempe, said an alternative explanation for black students’ test-score patterns could be that those students get an “artificial” boost the first year as they move out of troubled city schools. In the second year, students begin to fail and drop out of private schools, leaving only the most able students in private schools in the third year, he suggested.

“It shows all the signs of a randomized experiment gone bad,” he said.

Parents surveyed for that study, though, most often cited cost as the deciding factor for moving their students out of the private schools.

Still, the inconsistency of the two cities’ results should not discourage policymakers from continuing to experiment with voucher initiatives, said Mr. Gill of RAND.

“This is the best evidence we’ve got about the effectiveness of vouchers for students who attend private schools,” he said. “We need a lot more research to understand why these results would be true, and if they would be generally true.”

Related Tags:

A version of this article appeared in the February 27, 2002 edition of Education Week as Voucher Plans’ Test Data Yield Puzzling Trends


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Personalized Learning Webinar
No Time to Waste: Individualized Instruction Will Drive Change
Targeted support and intervention can boost student achievement. Join us to explore tutoring’s role in accelerating the turnaround. 
Content provided by Varsity Tutors for Schools
Student Well-Being K-12 Essentials Forum Social-Emotional Learning: Making It Meaningful
Join us for this event with educators and experts on the damage the pandemic did to academic and social and emotional well-being.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Privacy & Security Webinar
K-12 Cybersecurity in the Real World: Lessons Learned & How to Protect Your School
Gain an expert understanding of how school districts can improve their cyber resilience and get ahead of cybersecurity challenges and threats.
Content provided by Microsoft

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion Rebooting Assessment and Accountability Post-Pandemic: What Now?
The disruptions of the pandemic have made this an ideal time to rethink accountability and assessment.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Assessment Don’t Use State Tests ‘Punitively,’ Ed. Secretary Cardona Warns
As federal accountability restarts after two years, guidance from the department underscores how complicated that could be.
5 min read
Image of data, target goals, and gaining ground.
Assessment Latest Round of Federal Grants Aims to Make States' Assessments More Equitable, Precise
The U.S. Department of Education awarded over $29 million in competitive grants to 10 state education agencies.
2 min read
Assessment review data 599911460
Assessment Opinion Are There Better Ways Than Standardized Tests to Assess Students? Educators Think So
Student portfolios and school community surveys are but two of the many alternatives to standardized tests.
3 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)