Dozens of studies have come out over the past months concluding that the pandemic had a negative—and uneven—effect on student learning.
National analyses have shown that students who were already struggling fell further behind than their peers, and that Black and Latino students experienced greater declines in test scores than their peers.
But taken together, what implications do they have for school and district leaders looking for a path forward?
Here are four questions and answers, based on what we’ve learned from the most salient studies, that dig into the evidence.
Did students who stayed in remote learning longer fare worse than those who learned in person?
Generally, yes—but not in every single instance.
School buildings shut down in spring 2020. By fall 2021, most students were back learning in person. But schools took a variety of different approaches in the middle, during the 2020-21 school year.
Several studies have attempted to examine the effects of the choices that districts made during that time period. And they found that students who were mostly in-person fared better than students who were mostly remote.
An analysis of 2021 spring state test data across 12 states found that districts that offered more access to in-person options saw smaller declines in math and reading scores than districts that offered less access. In reading, the effect was much larger in districts with a higher share of Black and Hispanic students.
Assessment experts, as well as the researchers, have urged caution about these results, noting that it’s hard to draw conclusions from results on spring 2021 state tests, given low rates of participation and other factors that affected how the tests were administered.
But it wasn’t just state test scores that were affected. Interim test scores—the more-frequent assessments that schools give throughout the year—saw declines too.
Another study examined scores on the Measures of Academic Progress assessment, or MAP, an interim test developed by NWEA, a nonprofit assessment provider. Researchers at NWEA, the American Institutes for Research, and Harvard examined data from 2.1 million students during the 2020-21 school year.
Students in districts that were remote during this period had lower achievement growth than students in districts that offered in-person learning. The effects were most substantial for high-poverty schools in remote learning districts.
Still, other research introduces some caveats.
The Education Recovery Scorecard, a collaboration between researchers at Stanford and Harvard, analyzed states’ scores on the 2022 National Assessment of Educational Progress. They compared these scores to the average amount of time that a district in the state spent in remote learning.
For the most part, this analysis confirmed the findings of previous research: In states where districts were remote longer, student achievement was worse. But there were also some outliers, like California. There, students saw smaller declines in math than average, even though the state had the highest closure rates on average. The researchers also noted that even among districts that spent the same amount of time in 2020-21 in remote learning, there were differences in achievement declines.
Are there other factors that could have contributed to these declines?
It’s probable. Remote learning didn’t take place in a vacuum, as educators and experts have repeatedly pointed out. But there’s not a lot of empirical evidence on this question just yet.
Children switched to virtual instruction as the pandemic unfolded around them—parents lost jobs, family members fell sick and died. In many cases, the school districts that chose remote learning served communities that also suffered some of the highest mortality rates from COVID.
The NWEA, AIR, and Harvard researchers—the group that looked at interim test data—note this. “It is possible that the relationships we have observed are not entirely causal, that family stress in the districts that remained remote both caused the decline in achievement and drove school officials to keep school buildings closed,” they wrote.
The Education Recovery Scorecard team plans to investigate the effects of other factors in future research, “such as COVID death rates, broadband connectivity, the predominant industries of employment and occupations for parents in the school district.”
Most of this data is from the 2020-21 school year. What’s happening now? Are students making progress?
They are—but it’s unevenly distributed.
NWEA, the interim assessment provider, recently analyzed test data from spring 2022. They found that student academic progress during the 2021-22 school did start to rebound.
But even though students at both ends of the distribution are making academic progress, lower-scoring students are making gains at a slower rate than higher-scoring students.
“It’s kind of a double whammy. Lower-achieving students were harder hit in that initial phase of the pandemic, and they’re not achieving as steadily,” Karyn Lewis, the lead author of the brief, said earlier in November.
What should schools do in response? How can they know where to focus their efforts?
That depends on what your own data show—though it’s a good bet that focusing on math, especially for kids who were already struggling, is a good place to start.
Test results across the board, from the NAEP to interim assessment data, show that declines have been larger in math than in reading. And kids who were already struggling fell further behind than their peers, widening gaps with higher-achieving students.
But these sweeping analyses don’t tell individual teachers, or even districts, what their specific students need. That may look different from school to school.
“One of the things we found is that even within a district, there is variability,” Sean Reardon, a professor of poverty and inequality in education at Stanford University and a researcher on the Education Recovery Scorecard, said in a statement.
“School districts are the first line of action to help children catch up. The better they know about the patterns of learning loss, the more they’re going to be able to target their resources effectively to reduce educational inequality of opportunity and help children and communities thrive,” he said.
Experts have emphasized two main suggestions in interviews with Education Week.
- Figure out where students are. Teachers and school leaders can examine interim test data from classrooms or, for a more real-time analysis, samples of student work. These classroom-level data are more useful for targeting instruction than top-line state test results or NAEP scores, experts say.
- Districts should make sure that the students who have been disproportionately affected by pandemic disruptions are prioritized for support.
“The implication for district leaders isn’t just, ‘am I offering the right kinds of opportunities [for academic recovery]?’” Lewis said earlier this month. “But also, ‘am I offering them to the students who have been harmed most?’”
