The nation’s largest tax-credit scholarship program doesn’t appear to be hurting the academics of students who remain in public schools, a new study shows.
Those students who remained in public schools during the expansion of Florida’s tax-credit-funded private school vouchers program—the nation’s largest, with more than 100,000 students participating— actually saw improvements in their reading and math test scores, and were on average suspended less often and absent less often, concludes the study, which was recently published by the National Bureau of Economic Research.
And the most likely explanation for those gains, the study says, appears to confirm one of the arguments made by private school choice boosters: The competitive pressure that comes from students having a lot of school choices led the public schools to improve their offerings.
In all, the study is among the first to examine the long-term effects of a choice program. Most existing studies look only a few years after such a program has been put into place, though they have largely found small benefits.
“It’s definitely notable and important to say you continue to see consistent, modest improvement as the program expands in size,” said Cassandra Hart, one of three researchers on the study. She is an associate professor at the school of education at the University of California, Davis.
While the study can’t resolve some of the deeper ideological questions surrounding these types of programs—the financial pressure of losing students to private schools is real—it does suggest that some other concerns aren’t yet warranted.
It’s long been a concern, for example, that the most motivated families would benefit from such a program—concentrating the neediest students behind them in public schools. But that does not appear to be the case in Florida.
That said, the researchers cautioned, it isn’t clear when Florida’s expansion of this program could reach a “tipping point,”
“Theoretical models would also say that at some point you should see some declines, because the positive achievement effect is outweighed by things like limited resources,” said Krzysztof Karbownik of Emory University, one of Hart’s partners on the study. “So it’s very much an empirical question: Where is this tipping point?”
Competitive Pressures
Hart and Karbownik teamed up with David N. Figlio of Northwestern University for the study. It’s based on a sample of more than 1 million Florida students. The researchers merged Florida birth records with test-score data from grades 3-8 from the 2002-03 through the 2016-17 academic years. Through 2011-12, they also looked at each student’s records of suspensions or absences.
Building on some prior work of Figlio and Hart’s, the researchers developed five measures of school competition: How many private schools in the same grade levels each school was near, the number of types of private schools that were nearby, how many churches and religious schools were nearby, and the proportion of students served in private schools, among others. Then they analyzed student outcomes for their relationship to these competitive pressures..
The findings showed a statistically significant relationship for nearly all the measures studied, with the students attending schools in areas with more competition seeing greater increases in reading and math test scores. And, in one of the very first pieces of evidence on the effects of these programs beyond academics, the researchers found reductions in both suspensions and absences, suggesting that student behavior also improved.
Importantly, these results weren’t uniform among all students and schools: Students from low-income families benefited more as schools responded to the tax-credit programs. Reductions in suspensions were most dramatic for Hispanic students.
Here’s one of the best ways to picture these effects. This diagram from the study shows the effects for all students; for those who at one point received free- or reduced-price lunches, a proxy for poverty; and for those who never qualified for that program. Don’t worry too much about the finer-grained details and look at the overall patterns: They are largely positive and growing for academic effects, while suspensions and absences fall. And note that those patterns are stronger for economically disadvantaged students than for their more-advantaged peers.
Generally speaking, these are not enormous gains. The researchers characterized them as modest. But it’s noteworthy that they are there, and that they seem to be increasing as the program expanded.
Importantly, the researchers tried to rule out alternative explanations for the findings. One is that the composition of students left in the school differed drastically from what it looked like before the program started. But the researchers found that those students who stayed were not, on average, higher- or lower-performing students. Second, they theorized, the smaller class sizes could account for the findings, since much research shows a connection between smaller classes and learning. But the researchers found that class-size benefits only contributed a small portion of the overall effects.
The most likely explanation, they said, are instructional changes within the schools themselves.
Putting Findings in Context
One of the reasons the study matters is because so much of the research literature on private school choice programs has been anything but clear-cut. (See this Chalkbeat explainer for more.)
By and large, the track record on the performance of students who receive vouchers is murky. The most recent studies consistently find negative effects, not positive ones, on the test scores of students who used them to attend other schools. (Longer-term impacts, on high school graduation and college matriculation, are more mixed and seem to depend on program context, Chalkbeat noted.)
And what about the effects on public school students? Most of those studies have focused on the first few years after a private school choice program’s introduction. Those have mainly found small positive effects. Florida is a different story. The program is more than 15 years old and now serves more than 108,000 students, making it the single largest tax-credit program in the nation. It has also morphed considerably over this time period. Initially restricted to families with incomes just slightly above the federal poverty line, it now serves families with incomes up to 260 percent of the poverty line—or more than two-and-a-half times.
“It does a better job than any study I’ve seen to date of asking how the existence of a large voucher program affects students who don’t use it,” said Martin West, a professor of education at the Harvard Graduate School of Education. (He did not contribute to the study.)
That’s an important addition in part because of the conventional wisdom on these programs, West added. “Many people assume that voucher programs have negative consequences for public school systems, and the students who make use of them. Even treating that as an empirical question, rather than a given, is important.”
The study doesn’t directly address value-laden disputes over voucher programs. Those longstanding arguments include whether the money that funds them should be considered public. (Many argue that it should, since they’re funded through tax credits, but Florida courts have ruled that its program does not involve public money.)
More recently, Florida’s program has come under scrutiny because some of the schools families have selected contain explicitly anti-LGBTQ policies, and because there is far less accountability for results than in the state’s public schools.
But Can It Be Replicated?
The researchers did sound this note of caution: Don’t rush to the conclusion that any old private school choice program will have these same kinds of effects. The Florida program is unique in a number of ways, particularly its duration and its changing features over time.
And the study wasn’t designed to peer inside the “black box” of schools to determine precisely what happened; it tells us the “what” but not much about the “how.” We can’t determine, for example, if these are genuine improvements in learning or, in contrast, whether the public schools doubled down on test-prep or some other strategy to boost scores. The improvement on non-academic outcomes, though, suggests overall climate did improve for students.
It’s clear, too, that individual families have very different experiences with these programs. (My colleague Arianna Prothero wrote about the struggles of one family, who used two of the state’s other private school choice programs, although not the one this study is based on.)
The researchers themselves said they’d welcome some sociological, qualitative studies to help tease out some of those details.
“I would love to see the results of this being district offices and researchers getting a better sense of what’s happening in schools that’s driving these effects,” Hart said. “I think that would be extremely valuable for district offices to know—and not just in the context of what’s happening in voucher schools, but in the search of best practices that can help all students.”
And finally, the study doesn’t say anything about the financial impacts that these public schools may have grappled with even as they apparently served students better. It’s reasonable to assume that these, too, would differ in Florida from other states. (You probably have more cushioning in your budget as a large countywide district in Florida than as a tiny district in Indiana.)
Photo: Erica Florea and daughter, Jessica, 14, at home in Jupiter, Fla. The family had a difficult experience with the state’s private school choice programs.—Josh Ritchie for Education Week