Assessment Q&A

Q&A: Top NAEP Official Talks About the Future of Nation’s Report Card and More

By Sarah D. Sparks — September 11, 2023 5 min read
Photo of students taking exam on laptops.
  • Save to favorites
  • Print

U.S. students have seen unprecedented declines in math and reading performance since the pandemic on the National Assessment of Educational Progress.

Peggy Carr, the commissioner for the National Center for Education Statistics, which administers the assessment known as the Nation’s Report Card, said the NAEP system itself is changing to better understand how the pandemic has affected students and how they can regain their academic momentum.

“Everything has been in turmoil since COVID—it’s just complete chaos for almost any industry you can think of—and education really took the brunt of it,” she said. “People don’t realize what it means when your education system globally is not in place and it impacts everything else, so we have been busy as soon as we could get in the field collecting data.”

In addition to the 2024 NAEP, now in the field, NCES is launching the first post-pandemic cohorts for its long-term studies of early childhood (via the Early Childhood Longitudinal Study-Kindergarten) and the high school longitudinal study. Both collections track groups of students over years, gathering data on their academic progress and social development, as well as background information from parents and teachers. Carr spoke with Education Week about what her agency is learning from post-pandemic assessments.

This interview has been edited for length and clarity.

What surprised you in the NAEP results since the pandemic?

Carr: Many things that have emerged as impacts of the whole COVID situation we could have predicted: [poorer] mental health, bullying is up, all those things. But I don’t think I would’ve predicted that sustained chronic absenteeismis something we now need to start worrying about. [The U.S. Education Department defines chronic absenteeism as missing 15 or more school days in a year; NAEP also measures absenteeism by the number of students who have missed three days or more in the month before testing.] We have been asking about [chronic absenteeism] prior to the pandemic, so we have a gauge of what it was like prior to the pandemic, and now we can see what it’s like three years out, and it’s up.

And ironically, I wouldn’t have predicted this either, but we’ve found regardless of the depth of the decline [in student performance on NAEP]—whether it be a 16-point drop or a 5-point drop—it looks like somewhere between one and a half and two points of the drop [in average NAEP performance] can be associated with an increase in chronic absenteeism.

Every state saw at least some declines in the NAEP during COVID. But, why do you think some states declined so much more than others?

Carr: There’re so many other factors that seem to have influenced how states did … and not just whether they were in or out of school—like whether or not they had technology in place and the infrastructure to support instruction when things hit the fan.

We did an analysis of schools [in a forthcoming study with the American Institutes of Research] that we classified into different groups based upon what their health environment looked like when COVID hit—you know, death rates, teenage pregnancy, any health-related statistics that help you understand what kind of struggles a particular community is going through. We classified them based upon a healthier environment for the schools, or a less healthy environment for the schools.

And what we found out is that schools that already had these determinants that made their environment, their social and health environment, more challenging showed more declines—steeper declines—than those schools that were in environments that were physically healthier. In fact, those that had less-healthy environments showed more double-digit declines.

There has been a big increase in the number of students performing below the ‘basic’ achievement level in the on NAEP, a group we historically haven’t known as much about. What are you finding out about these students?

Carr: Actually, prior to COVID, we were seeing the bottom drop and the students at the top were holding their own or improving. There was this bifurcation with more students at the bottom falling steeper, with lower scores. And then after COVID, it was clearly exacerbated. So the problem that we already had now is a steeper, more troublesome problem.

The average score for Black students, for example, now is in that below-basic category. Right? We know what they don’t know and what they can’t do. But we don’t have a lot of [test] items to help us understand what they are able to do to be able to have something that’s more actionable.

How do you get a better picture of those lowest-performing students?

Carr: Back in the mid-to-late 2000s, No Child Left Behind required that Puerto Rico take the [NAEP] assessment too, in math, but we were struggling to get them on the scale because they were so low, there weren’t enough [test] items down there at the bottom. We created back in that time period, what they call KSA items—knowledge, skill, and ability items—in which we took out the [test] item distractors that really weren’t related to math. We took those out and those items were more accessible to the students in Puerto Rico.

But you know what, it wasn’t just Puerto Rico; it was Detroit and other low-performing districts that benefited from us putting those kinds of items on the assessment. So we’re using that as a model moving forward today to better populate the bottom of the scale, because there are so many kids down there.

The pandemic was the first time in NAEP’s history that it couldn’t be administered because of the shutdowns. How has that changed NAEP?

Carr: There is a concern that if something like this happens again, would we be able to test students from wherever we are? We actually already were planning online testing, but after COVID, it was clear we needed to speed it up. So what will happen in ’24 is we’ll have all of the content on devices, but we’re transitioning to the cloud as well so the content can be streamed down and streamed up [so that students could participate in the NAEP via whatever device schools have available, rather than using tablets NAEP brings in person]. That will make us far more flexible, if something like this happens again.

We’re going to transition from our [Surface tablet] devices, to use devices that are in the schools [like] Chromebooks, which can be found in 70 percent or more of schools. We’re going to do a bridge study in which comparable samples of students will take the assessment on Chromebooks, and others will take it with Surface Pros, which is what we’re doing now, and any differences, we’ll be able to statistically remove from our trends moving forward.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Budget & Finance Webinar
Innovative Funding Models: A Deep Dive into Public-Private Partnerships
Discover how innovative funding models drive educational projects forward. Join us for insights into effective PPP implementation.
Content provided by Follett Learning
Budget & Finance Webinar Staffing Schools After ESSER: What School and District Leaders Need to Know
Join our newsroom for insights on investing in critical student support positions as pandemic funds expire.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
How can districts build sustainable tutoring models before the money runs out?
District leaders, low on funds, must decide: broad support for all or deep interventions for few? Let's discuss maximizing tutoring resources.
Content provided by Varsity Tutors for Schools

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment What the Research Says AI and Other Tech Can Power Better Testing. Can Teachers Use the New Tools?
Assessment experts call for better educator supports for technology use.
3 min read
Illustration of papers and magnifying glass
iStock / Getty Images Plus
Assessment What the Research Says What Teachers Should Know About Integrating Formative Assessment With Instruction
Teachers need to understand how tests fit into their larger instructional practice, experts say.
3 min read
Students with raised hands.
E+ / Getty
Assessment AI May Be Coming for Standardized Testing
An international test may offer clues on how AI can help create better assessments.
4 min read
online test checklist 1610418898 brightspot
champpixs/iStock/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Design for Improvement: The Case for a New Accountability System
Assessments in more frequent intervals provide useful feedback on what students actually study. New curriculum-aligned assessments can le...
Content provided by Cognia