Opinion
School Choice & Charters Commentary

Voucher and Class-Size Research

By Alex Molnar & Charles Achilles — October 25, 2000 10 min read
There is no comparison between class-size and voucher research.

The academic benefits of educational vouchers were blockbuster back-to-school news this autumn. The report that generated all the media attention, “Test- Score Effects of School Vouchers in Dayton, Ohio; New York City; and Washington, D.C.: Evidence From Randomized Field Trials,” was authored by a team of researchers, including Paul E. Peterson of Harvard University. Shortly after the voucher report was released, former U.S. Secretary of Labor Robert Reich wrote an opinion piece in The Wall Street Journal making a case for what he called “progressive vouchers.” Referring to the latest voucher study, Mr. Reich asserted: “Evidence mounts that vouchers do work for kids who use them.”

If by “work” Mr. Reich means that voucher programs increase academic achievement, perhaps he should review the evidence with more care. Virtually all of the data on the achievement impact of publicly funded voucher programs come from the Milwaukee Parental Choice Program and the Cleveland Scholarship and Tutoring Program. Although the Cleveland evaluation is ongoing, the Milwaukee data are now 5 years old. Moreover, the 1995 legislation that expanded the Milwaukee program to include religious schools also removed its evaluation component, so no new data will be available anytime soon. Therefore, most of the evidence about the impact of voucher programs will, for the foreseeable future, come from the privately funded voucher programs, such as those in Dayton, New York City, and Washington.

In the August 2000 report referred to by Mr. Reich, the Peterson team presented the results from Dayton, New York, and Washington averaged across grade levels. The combined averaged results from all three cities are also presented. Since averaged results tend to conceal inconsistent findings, they may make the achievement impact reported appear more generalized than it is.

In this case, even the averaged results are inconsistent. Considering the results that are significant at the conventional .05 level yields the following information: According to the Peterson team, simply being offered a voucher makes a significant difference in reading but not math achievement for African-Americans in New York City in year two. In Dayton, the offer produces no significant result in either year. In Washington, the offer makes a difference in African-American test scores in reading and math in both years. Actually switching to a voucher school in New York produces a significant advantage in reading for African-American students in both years, and in math in year one. In Washington, there is an advantage in math in both years, and in reading in year two only. In Dayton, switching to a voucher school produced no achievement advantage in either year.

Except in Washington, for which two grade-level groups (grades 2-5 and grades 6-8) for African-American students are provided, it is not possible to tell how students in different grades performed. The Washington results, however, appear to be driving the averaged findings for the three cities. More specifically, it seems that the averaged results are heavily influenced by African-American students in grades 2-5 in Washington.


The impression created by the averaged data was apparently troubling enough for Mr. Peterson’s New York research partners at Mathematica, a highly respected private research firm, to cause them to issue a separate statement, entitled “Voucher Claims of Success Are Premature in New York City.” In their statement, the Mathematica researchers reported that students offered a scholarship performed at about the same level as students in the control group. They further noted that the gains reported for African-American students in New York were largely the result of the scores of one group of students (who were in the 5th grade in 1998-99 and 6th grade in 1999-2000) and, therefore, must be interpreted with caution. The Mathematica statement pointed out that there was no achievement advantage for Hispanic students and advised that, since the gains reported were so concentrated in a single group, “one needs to be very cautious in setting policy based on the overall modest impacts on test scores.”

After reviewing the major research reports associated with the academic impact of vouchers, setting aside methodological qualms, and accepting at face value all findings that achieved the conventional .05 level of significance, we offer the following advice to parents who are concerned about the academic performance of their children and who are interested in vouchers:

  • In Milwaukee, enroll your child in a voucher school and wait three or four years. Your child may at that point begin to do better in math and reading (research by the Peterson group), or just in math (research by Princeton University’s Cynthia Rouse), or in neither subject (research by the University of Wisconsin-Milwaukee’s John Witte). A better choice would be to send your child to a Milwaukee public school with reduced class size. They outperform voucher schools (Rouse).
  • In Cleveland, enroll your child in a voucher school for 4th grade language arts and maybe science, depending on the classroom to which your child is assigned. Be careful, however, to avoid newly created private schools. Students in those schools perform significantly less well than students in the Cleveland public schools in all subjects tested (research by Indiana University’s Kim K. Metcalf).
  • In Washington, if you are an African-American parent, enroll your child in a voucher school in grades 2-5 (Peterson).
  • In New York City, if you are an African-American, enroll your child in a voucher school in 7th grade for the 2000-01 school year (Mathematica).

Even without considering the manifold methodological questions that threaten the validity of many voucher findings, the literature on vouchers and academic achievement adds up to an inconsistent gaggle of results that do not offer intelligible guidance to either parents or policymakers. Unfortunately, as Jay P. Greene previously has noted in Education Week (“Rescuing Education Research,” Commentary, April 29, 1998): “Interest groups have learned that they can successfully check research contrary to their goals by producing their own studies, no matter how lousy, to sow confusion among policymakers, journalists, and the attentive public about what to believe.”

Thus, despite the preliminary character and inconsistent outcomes of voucher research, well-financed voucher advocates are now sowing confusion by claiming that the achievement evidence is clear, systematic, and compelling. Some compare the power of the research and the magnitude of the voucher findings to the results of class-size research, most often the Tennessee Student-Teacher Achievement Ratio, or STAR, experiment. Since education policy is a matter of making choices among different policy options, that contention is worth examining.

In general, we can have the most confidence in a body of research that includes studies that are of sufficient size and scope to provide educationally important and reliable results. Moreover, current work should support the findings of earlier studies, and the findings should be the subject of rigorous peer review. Using these criteria, there is no comparison between class-size and voucher research.


The STAR project, conducted in Tennessee from 1985 to 1989, was a longitudinal, randomized experiment involving 11,600 students, grades K-3. STAR was designed to determine the effect of reducing class size in grades K-3 on student achievement and social development. At the end of the experiment, STAR students were followed throughout their K-12 education careers. They continue to be monitored in their postsecondary pursuits. Thus, STAR provides a large-scale, longitudinal assessment of the impact of reducing class size in the primary grades. Initial STAR results, showing a significant achievement advantage for students in small classes, and numerous follow-up and subsidiary studies, have been published in refereed journals.

The initial STAR data have been reanalyzed and the results confirmed by the Princeton University economist Alan Krueger. In a subsequent study, he also found a significant relationship between participation in a STAR small class and the rates at which students took college-entrance exams.

The literature on vouchers and achievement adds up to an inconsistent gaggle of results and doesn’t offer intelligible guidance to parents.

Follow-up studies of STAR students in grades 4-12 have found that they outperformed students who had been in regular-size classes on tests in all subject areas tested. In fact, students who had small classes from kindergarten to grade 3 outperformed regular-class students on all tests, every year, and the gap between small and regular-class test results increased each year. For example, an average STAR student in a small class for grades K-3 would outperform a regular-class student at grade 4 by 6.6 months and by 8.7 months at grade 8, fully five years after leaving the small-class environment.

The magnitude of positive voucher academic effects is sometimes described as comparable to the total STAR effects, rather than the STAR effects for minority students. For voucher studies, in which the overwhelming majority of students are African-American, the relevant comparison is to the performance of STAR minority students. For those students, the STAR effects were approximately double the total effects. Interestingly, it appears that the voucher students who perform at a higher level than students in public schools tend to attend smaller classes in smaller schools.

In addition to STAR, the positive results of class-size studies in, for example, Wisconsin; California; Fairfax County, Va.; and Burke County, N.C., complement the findings of powerful early-intervention studies. The Perry Preschool study followed 123 youngsters, and the Abecedarian study followed 109, from preschool into adulthood. The treatment-group students in these studies have consistently demonstrated better educational and social outcomes, such as employment, than their control-group peers. When the results of these early-intervention projects are considered in light of the STAR data on the enduring academic impact of small classes in grades K-3, all of the findings are strengthened, and policy implications can be drawn with considerable confidence.

It is now possible to say with a high degree of assurance that attendance in a small class is not a one-time inoculation that assures academic success. For long- term, continuing benefits, children should participate in small classes in the primary grades or preschool. The small-class experience must be intense—all day, every day, for a sufficient duration (that appears to be at least four years). When this combination of early intervention, intense treatment, and sufficient duration occurs, academic and social benefits continue to increase throughout schooling and extend into young adulthood.


It is no surprise that the recent RAND Corp. study of American education, “Improving Student Achievement,” found strong associations between student achievement and both higher public prekindergarten participation and reduced pupil-teacher ratios in the lower grades.

Some voucher advocates have taken to supporting their position by asserting that no one has demonstrated that vouchers harm public schools. In our view, this is not good enough. Good public policy should be constructed using the most powerful data available to actually help the public schools provide a high-quality education to the students they serve. When it comes to the evidence on vouchers and the evidence on class size, the policy choice is clear.


Alex Molnar is a professor of education and the director of the Center for Education, Research, Analysis, and Innovation at the University of Wisconsin-Milwaukee. He is a principal investigator of Wisconsin’s Student Achievement Guarantee in Education (SAGE) class-size-reduction program. Charles Achilles is a professor of education at Eastern Michigan University and was a principal investigator of the Tennessee Student Teacher Achievement Ratio (STAR) class-size experiment and numerous other class-size studies.

Related Tags:

Events

School & District Management Live Event Education Week Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School Choice & Charters Opinion Is Hybrid Home Schooling the Future of Education?
Rick Hess speaks with Mike McShane about hybrid home schooling, which combines the best of home schooling and traditional schooling.
7 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School Choice & Charters Oklahoma Charter Schools Granted Local Tax Revenue in 'Seismic' Settlement
A groundbreaking settlement will fundamentally change the way charter schools are funded in Oklahoma, despite vehement opposition.
Nuria Martinez-Keel, The Oklahoman
3 min read
This July 19, 2019 photo shows an Epic Charter Schools office in Oklahoma City. The Oklahoma State Board of Education voted Thursday in favor of an agreement with the state's public charter school association to settle a 2017 lawsuit.
This July 19, 2019 photo shows an Epic Charter Schools office in Oklahoma City. The Oklahoma State Board of Education voted Thursday in favor of an agreement with the state's public charter school association to settle a 2017 lawsuit.
Sue Ogrocki/AP
School Choice & Charters COVID-19 May Energize Push for School Choice in States. Where That Leads Is Unclear
The pandemic is driving legislators' interest in mechanisms like education savings accounts, but the growth may not be straightforward.
8 min read
Iowa Gov. Kim Reynolds delivers her Condition of the State address before a joint session of the Iowa Legislature on Jan. 12 at the statehouse in Des Moines, Iowa.
Iowa Gov. Kim Reynolds delivers her Condition of the State address to state lawmakers on Jan. 12. She's pushing a major school choice expansion.
Bryon Houlgrave/The Des Moines Register via AP
School Choice & Charters Letter to the Editor Are NOLA Charters a Mixed Bag?
To the Editor:
The opinion essay by Douglas N. Harris about how New Orleans’ education reforms post-Katrina are relevant to the COVID-19 era (“As Schools Recover After COVID-19, Look to New Orleans,” Sept. 30, 2020) highlights some basic improvements in the NOLA system but downplays the most significant aspects of those changes: the impact on people of color.
1 min read