Student Achievement

What’s Behind the COVID Academic Slide? Some Things Mattered More Than Remote Learning

By Sarah Schwartz — March 08, 2022 6 min read
Kindergarten teacher Karen Drolet works with a masked student at Raices Dual Language Academy, a public school in Central Falls, R.I., Wednesday, Feb. 9, 2022.
  • Save to favorites
  • Print

The past two years of the pandemic have brought an unrelenting stream of disruptions to regular schooling: building closures, quarantines, staff shortages. And study after study has shown that COVID-19 has slowed student achievement.

But a new analysis of student test data challenges the idea that remote learning alone caused the dips in academic progress, and suggests that other factors—beyond whether students were learning in-person or not—mattered more for kids’ growth during the 2020-21 school year.

The new study is from Curriculum Associates, a curriculum and assessment company. The analysis examines data from students who took the company’s diagnostic test, i-Ready, in reading and math.

The analysis found that there was only a small difference in student growth during the pandemic between students who were learning in person versus those learning remotely. That finding adds a new wrinkle to an emerging body of studies that found that students in remote learning did worse on summative state tests than students who learned in person last school year.

But when Curriculum Associates dug into those student groups further, other patterns emerged. Students who were further behind at the beginning of the 2020-21 school year had smaller academic gains than their peers. Among these students who were further behind, those in schools with mostly students of color and high percentages of low-income families made the least progress.

“The in-school/out-of-school [difference], that’s important, that’s a factor. But it’s more about where kids started,” said Matt Dawson, the director of efficacy and implementation research at Curriculum Associates, and the author of the study.

“Where students were before the pandemic is really, really important to take into account. The kids who were already struggling before the pandemic, they’re still struggling,” he added.

The data, and other reports on student achievement throughout the pandemic, present “actionable” information about which students need the most support, said Allison Socol, the assistant director of P-12 policy at EdTrust. Socol was not involved with the Curriculum Associates study.

“Educators have an opportunity right now, because they have this unprecedented investment in the form of the American Rescue Plan,” Socol said. “These findings really drive home the importance of using those additional resources to meet the needs of students who were already clearly in need of additional supports before the pandemic, and whose learning has been greatly impacted by the last two years.”

Students furthest behind lost the most ground last year

The Curriculum Associates study compared students’ growth before the pandemic to students’ growth during the 2020-21 and beginning of the 2021-22 school years. It reviewed data from about 2 million students who were tested in reading and about 2.4 million who were tested in math in grades K-8.

To determine whether test-takers were learning remotely or were in school buildings, Dawson looked at where students sat for the diagnostic tests, which could have been completed in a classroom setting or at home.

In general, he found, students were improving less during the 2020-21 school year than they had been prior to the pandemic. This effect was more pronounced in math than it was in reading. These results are in line with other analyses of interim tests from last school year.

But examining the test results by remote vs. in-person learning, there were only small differences in academic growth between students who were in person and students who were remote. In reading, remote students actually made slightly more growth than in-person students.

But the data don’t show that all students performed similarly during the pandemic. Differences emerged when Dawson looked at grade-level readiness.

Data from: Student Growth during COVID-19: Grade-Level Readiness Matters

The study took into account students’ initial placement level at the beginning of the 2020-21 school year. In reading, students who were on grade level stayed on track, continuing to improve throughout the pandemic—whether they were in-person or remote. But students who were two or more grade levels behind at the beginning of the year grew less than they had pre-COVID.

In math, all students—regardless of initial placement level—improved at a lower rate during the pandemic. But students who were two or more grade levels behind at the beginning of the 2020-21 school year saw the biggest slowdowns.

Data from: Student Growth during COVID-19: Grade-Level Readiness Matters

But even within this group—students who were two or more grade levels behind—some kids fared better than others during the 2020-21 school year. Students in majority-white suburban schools where a low percentage of students were from low-income families made more gains than students in schools in urban areas with mostly students of color and high percentages of low-income families.

The study results are more evidence that the pandemic exacerbated historical inequities, Dawson said, as these students were already further behind grade level before COVID.

One other important finding: Among 2nd graders who were behind in reading at the start of the 2020-21 school year, students who were learning remotely made more growth than students in person. This was true across demographic groups, though students in majority-white, higher-income, suburban schools made more progress.

‘Back to normal isn’t good enough’

Data showing that the pandemic hit the most vulnerable students the hardest isn’t new. But finding that there aren’t big differences in progress between students who learned remotely and students who learned in person departs from other recent studies, which show that in-person learners scored higher than their remote peers on spring 2021 state tests.

In part, this is because the Curriculum Associates study was examining changes in student growth, rather than a change in average scores, Dawson said. The Curriculum Associates data is also tracking the same groups of students pre-COVID and during COVID, while these other studies compare different children, he added.

Still, Christine Pitts, a resident policy fellow at the Center on Reinventing Public Education who was not involved with the study, questioned Curriculum Associates’ method of using testing location as a proxy for whether students were learning remotely or in person.

“Whether or not they took the test remotely may not actually be the best indicator of whether they were learning remotely,” she said. It’s possible that a student could have taken a diagnostic in class, and then switched to remote learning for the next three months, she said.

The study acknowledges this limitation, but notes, “we believe these deviations occurred at random and do not bias the sample in one direction or another.”

Now that the vast majority of students are in school buildings, though, determining the effectiveness of schools’ previous remote learning responses is a secondary priority to using data on student needs to plan for the future, Pitts said.

“It’s really important for us to see these results and immediately start to make sense of them in the context of the state budget planning right now,” she said. There are resources that state and district leaders can use to identify which interventions will have the greatest impact, she said, referencing a recent analysis from researchers at NWEA.

This kind of national-level data is useful, Socol added, but districts should be diving into their local data in the same way—breaking it down into student groups “to drive decisions about how to allocate resources,” she said.

Dawson, too, said that the Curriculum Associates data suggests that schools will need to offer more supports than they have in the past, in order to accelerate student learning: “Back to normal isn’t good enough.”

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
Future of the First Amendment:Exploring Trends in High School Students’ Views of Free Speech
Learn how educators are navigating student free speech issues and addressing controversial topics like gender and race in the classroom.
Content provided by The John S. and James L. Knight Foundation
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Start Strong With Solid SEL Implementation: Success Strategies for the New School Year
Join Satchel Pulse to learn why implementing a solid SEL program at the beginning of the year will deliver maximum impact to your students.
Content provided by Satchel Pulse
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Webinar
Real-World Problem Solving: How Invention Education Drives Student Learning
Hear from student inventors and K-12 teachers about how invention education enhances learning, opens minds, and preps students for the future.
Content provided by The Lemelson Foundation

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Achievement From Our Research Center Academics or Fun? Principals, After-School Providers Differ on Priorities
Principals are twice as likely as after-school program providers to say academics should be the top priority.
4 min read
Kyengye Asende (left) and Donat Jean, both sixth-grade students at John P. Fishwick Middle School, celebrate completing a worksheet during an after school program for English learners at the middle school on February 14, 2022, in Roanoke, Va.
Kyengye Asende, left, and Donat Jean, both 6th grade students at John P. Fishwick Middle School in Roanoke, Va., celebrate completing a worksheet during an after-school program in February for English language learners.
Heather Rousseau/The Roanoke Times via AP
Student Achievement Opinion The Pandemic’s Toll on Academic Growth Wasn’t Uniform. Recovery Efforts Can’t Be Either
Returning to "normal" will not be enough next school year, write three researchers.
Dan Goldhaber, Andrew Mceachin & Emily Morton
5 min read
Conceptual illustration: Some students were more impacted by pandemic
Vanessa Solis/Education Week and iStock/Getty
Student Achievement With So Many Kids Struggling in School, Experts Call for Revamping 'Early Warning Systems'
A new collaborative says the indicators developed to flag students at risk of not graduating need an update for post-pandemic schools.
4 min read
Woman sitting at a desk in front of a monitor showing a database of different types of charts.
DigitalVision/Vectors/Getty
Student Achievement What the Research Says Pace of Grade Inflation Picked Up During the Pandemic, Study Says
But the ACT found that the higher grades were not reflected in college-admission test scores.
4 min read
A+ in chalk on a traditional blackboard.
E+/Getty