At least 29 states and the District of Columbia have passed legislation over the past decade aimed at improving early reading instruction and student outcomes. But have these laws actually moved the needle on kids’ achievement?
A new study from researchers at Michigan State University examines the question—and finds that the outcomes are mixed.
In general, these policies are linked to improvements on states’ year-end standardized tests. But only states with comprehensive policies—legislation that provided support, training, and funding for instructional change, and implemented 3rd grade retention policies—also saw gains on the National Assessment of Educational Progress, the test known as the “Nation’s Report Card.”
Those states also made bigger jumps on standardized tests scores than their counterparts with less comprehensive policies.
The findings are especially relevant now, as the “science of reading” movement has gained steam in state legislatures. In the past two years alone, 18 states passed new laws requiring schools to use evidence-based instructional approaches and materials, provide new training for teachers, or offer interventions for struggling students.
This research has implications for states that are passing new policies or reshaping existing ones, said Amy Cummings, a doctoral student in education policy at Michigan State University, and one of the authors of the paper.
“Having these comprehensive supports for teachers and students does appear to be really important,” she said. “Retention is the thing that gets a lot of attention in the media, and a lot of previous research has hinged on the retention component of these policies. … But what we’re seeing is an effect over and above retention of including all these other supports.”
Umut Özek, a senior economist at the RAND Corporation who studies education policy, said the paper is “important work” for understanding the effects of these laws on a broad scale.
“Up until now, we’ve had these individual studies looking at the effects of early literacy interventions, and in part these early grade retention policies in different states,” said Özek, who was not involved with this research. “But this study provides a more comprehensive look at the national level.”
The ingredients of a ‘comprehensive’ policy
The research, which was presented at the American Educational Research Association conference earlier this month, has not yet been peer reviewed.
To categorize early literacy policies as “comprehensive” or not, the researchers used criteria developed for a 2021 analysis by ExcelinEd, an advocacy group founded by Jeb Bush, Florida’s former governor. Comprehensive policies met all criteria, including training and coaching for teachers, funding, and lots of supports for struggling students. All comprehensive policies also included retention components. (See all of the criteria in Table 1 in the paper, here.)
Then the researchers used the Stanford Education Data Archive, which aggregates yearly data on state reading and math tests, to analyze the states’ performance on state tests from 2009 through 2018. They also examined NAEP results—a test that isn’t tied to consequences for students, teachers, or schools.
“High-stakes test scores are often criticized because they might capture other things than student learning,” like teaching to the test, said John Westall, a postdoctoral research fellow at Michigan State University, and one of the authors on the paper.
Because NAEP isn’t attached to retention or accountability standards in the same way, he said, “we think it does a better job of measuring the actual skills of the students in the subject that’s being tested.”
Overall, states with any kind of literacy policy saw gains in high-stakes reading tests in grades 3-5, with the biggest gains in 3rd grade. Students improved by 7 to 14 percent of the annual academic growth of a typical 3rd grader.
“These states that have the retention requirement tend to have bigger test-score increases than states that don’t have retention requirements—but not as big as states that have these comprehensive policies, which include retention, but also these other supports for teachers and students,” Westall said.
They also found that these policies didn’t widen socioeconomic or racial achievement gaps on these tests, and suggestive evidence that the policies slightly reduce disparities.
States with comprehensive policies also saw modest, but significant, improvements on students’ NAEP scores—while states with policies that weren’t comprehensive did not.
The comprehensive policies also seemed to lead to more sustained achievement effects on the state tests, Westall said. Students in these states continued to improve at above-average rates through 8th grade, while gains for students in states with less comprehensive policies faded out after 5th grade.
Why ‘business as usual’ doesn’t work
Of course, early literacy legislation wasn’t the only policy change happening during the years that the researchers examined.
In the early 2010s, some states also began receiving funding through Race to the Top—a competitive federal grant program that offered funding for states that agreed to embrace certain policies, including the adoption of rigorous, common standards in English/language arts and math.
The researchers separately analyzed states that got these funds and those that didn’t, and found that states with early literacy policies that had not received Race to the Top funding still saw significant improvements on high-stakes test scores. These effects were similar in size to states that had both early literacy policies and received Race to the Top money.
Overall, the study’s findings are consistent with previous research on 3rd grade retention policies that shows holding students back is most effective when it’s coupled with additional support, Özek said.
“The main objective of these retention policies is to give these kids extra time to catch up. But whether they can catch up or not depends on how well you use that time, that additional year,” he said. “If it’s business as usual, it’s less likely that these kids will benefit from these policies.”
