Students have made some progress toward academic recovery, but overall achievement hasn’t yet reached pre-pandemic levels, new test results show.
The data are from the COVID-19 School Data Hub, a 2021 project launched by Emily Oster, a professor of economics at Brown University. The hub has recently aggregated 2023 student test-score results across grades 3-8 in math and reading for about half of all states.
The results show that most states have made up some ground in math, compared to the 2020-21 school year. But in reading, some states are making progress while others have regressed. Only a few states have recovered to pre-pandemic levels: Iowa and Mississippi in both math and ELA, and South Carolina and Tennessee in ELA.
“For us, the main takeaway has been the dramatically different recovery patterns between English/language arts and math,” said Clare Halloran, the associate director of the COVID-19 School Data Hub. “That was surprising.”
The group of states includes those that did not make changes to their assessments in 2021 or 2022, and have already released scores this year. The hub team plans to update the collection results as more states release their 2023 scores.
Testing experts say it’s hard to know exactly how to interpret these results.
“The overall story that there was a negative impact [of the pandemic] is not too controversial. Once you get past that, it’s much more controversial,” said Derek Briggs, a professor in the Research and Evaluation Methodology program at the University of Colorado Boulder.
Because different states design their tests differently, it can be difficult to compare student progress across locations. It’s also hard to know what exactly caused score jumps in certain states.
Education Week spoke with several testing experts about how to interpret the new state assessment data. Here are five considerations they raised.
Why ELA and math scores might be different
The data show that students are progressing more steadily in math than they are in English/language arts. In ELA, states have vastly different trends from one another. Testing experts say there are a few reasons this might be the case.
Math tests are “more sensitive to changes in instruction,” said Marianne Perie, the director of assessment research and innovation at WestEd. Previous research has shown that it’s easier to move math scores up with consistent teaching than it is to increase ELA scores.
Some curriculum experts have criticized reading tests for ultimately being a measure of how much students know rather than how well they read.
ELA tests also measure a combination of discrete skills—reading, writing, and speaking/listening, said Andrew Ho, a professor at the Harvard Graduate School of Education. States make different choices about how to balance and evaluate the combination of these skills in their tests.
So if students got less practice with writing during the pandemic, for example, that could result in big negative effects on some states’ tests—but smaller effects on others.
“Because different states measure in different ways, measurement experts have been a little wary of accurate measurement of progress in ELA through the pandemic,” Ho said.
Students in different grade levels could be recovering at different rates
The state test results that are reported in the COVID-19 School Data Hub are aggregated across all grades, 3-8.
“I think that can mask certain things,” said Benjamin Shear, an assistant professor in the Research and Evaluation Methodology program at the University of Colorado Boulder, referencing differences by grade level.
For example, in a few of the states Perie works with, 3rd grade ELA scores specifically are down. Most of these students were learning online in 1st grade, an important year for early reading foundations.
“Does that have more of an impact than being online in kindergarten?” Perie asked.
It’s possible, she said, that certain cohorts of students—who were online during key moments in their educational experience—might see more persistent score declines than others.
We don’t know exactly why some states are outliers
A few states stood out in the data, where students had reached or exceeded pre-pandemic achievement levels. This was the case in Iowa and Mississippi in both subjects, and in Tennessee and South Carolina in reading.
It’s hard to know exactly why some states are seeing these strong results, said Briggs.
“Just because you see what looks like a trend, then attributing that trend to a change in policy [like a specific pandemic-recovery effort] is really, really hard to do,” he said.
Still, states have claimed that certain pandemic-era choices relate to their outcomes. In a press release about 2023 test scores, the Iowa Department of Education Director McKenzie Snow cited the state keeping school buildings open as a reason why students experienced a smaller decline in test scores during the pandemic than other states.
“There is some pretty strong national evidence that states who sent their kids back to school sooner had fewer issues,” said Perie. Even so, other states where many districts reopened for the 2020-21 school year—such as Arkansas—aren’t making similar academic progress.
Hopefully, the data can shine a spotlight on states that are seeing a lot of progress, said Halloran, “to help explore those areas a little more.”
She’d like to know how new state laws around literacy instruction might be affecting results. “Many states have implemented a lot of new … policies and approaches and programs,” she said. “There’s a lot of room for exciting research there.”
Trends between student groups vary
Many other analyses of student test data—from the interim assessments that students take a few times a year to the federally administered National Assessment of Educational Progress—have shown the uneven toll the pandemic took on student achievement. Students from low-income backgrounds fared worse than their peers, and students who were already behind saw widening gaps.
This analysis also shows that in many states, the gap in student proficiency between high-income and low-income districts is increasing.
“The real story of the pandemic has been less about decline and recovery than it has been about inequality, inequality, inequality,” said Ho.
Still, he said, measuring gaps solely by using changes in the percent of students proficient could distort those comparisons.
The reason why is technical, but important. In some situations, small changes in students’ scores could tip a lot of students over the edge from not proficient to proficient—or vice versa. It could look like a state was making big gains or losses, when in reality student scores only changed a small amount.
Analysts should consider other metrics as well, like average scale scores, that can provide a more precise understanding of gaps between student groups, Ho said.
Comparing anything to 2021 data comes with caveats
The state testing landscape in 2021 looked a lot different than in previous years.
In some states, such as Colorado, not every grade was tested in every subject. Across the country, student participation was much lower than usual.
“2021 has a lot of error around it because so many states didn’t get strong participation,” said Perie, noting that school enrollment rates changed during this time period, too.
All of these factors make it possible that the population of students who participated in testing between 2021 and 2023 changed in such a way that skews the comparison, said Shear.
Oster, Halloran, and their team took this variability in 2021 into account. For states in which 70 percent of students or fewer participated in testing in 2021—including Colorado, Delaware, Michigan, Oregon, and Nevada—they used 2022 results as their first post-pandemic measure.
Briggs said it could be useful to juxtapose state test results with other data, such as the state-level NAEP results that are released every two years.
