The next Nation’s Report Card will take a more nuanced look than previous assessments ever have at how students’ socioeconomic status affects their academic achievement.
The 2024 National Assessment of Educational Progress, due out in late December or early January, will introduce a new composite measure of student income that takes into account broader family and school resources.
The new measure uses an index incorporating:
- the student’s eligibility for school meals and other federal safety-net programs, such as Temporary Assistance to Needy Families;
- the total share of students eligible for income-related programs at the school;
- the number of printed books in the student’s home; and,
- for grades 8 and 12, the education level of either parent. (Fourth graders have not been able to identify their parents’ education consistently.)
The index allows test scores to be disaggregated for high-, middle-, and low-income students. International testing groups such as the Organization for Economic Cooperation and Development (which administers the Program for International Student Assessment) use similar poverty indexes, but NAEP’s is designed to be more stable over time.
“We’re excited about this,” said Dan McGrath, acting associate commissioner for assessments of the National Center for Education Statistics, which administers the NAEP. “What is provided with NAEP is this more nuanced variable ... that you can use to look at relationships between student achievement and students of different socioeconomic backgrounds state by state.”
Because the index uses data that NAEP already collects from school or student surveys, researchers and policymakers also will be able to retroactively apply the new poverty measures to some 20 years of student data.
For example, one American Institutes for Research study used a similar index to analyze achievement gap trends between the highest and lowest-income students from 2003-2017. The index data showed more of the poorest 20 percent of 8th graders in each state reached basic and proficient math achievement over that time, but 14 states saw widening achievement gaps between high- and low- income students.
Using the poverty index and English-language proficiency alone explained all of the achievement gaps between white and Hispanic students, and most of the gaps between white and Black students, said Markus Broer, a managing researcher at AIR and director of the poverty index project for NAEP.
Using a poverty index, Broer said, “You can understand better the challenges which might only involve a small percentage of students, but students with very low [socioeconomic status],” he said. “I hope it can help shed light on different areas of the whole SES spectrum.”
Beyond a one-dimensional measure
NAEP traditionally has analyzed test scores based on whether a student is eligible for federal free or reduced-price meals. But beginning in the 2011-12 school year, the “community eligibility provision” has allowed schools serving high concentrations of low-income students to offer free lunch schoolwide.
While community eligibility dramatically increases the number of students who can get access to stable food at school, it dilutes the usefulness of the education field’s most broadly used proxy for poverty. The Urban Institute found that moving from individual to community eligibility can undermine the ability to understand student achievement for low-income students.
In West Virginia, for example, the share of students reported as eligible for school meals jumped from 46 percent in 2011, when the state adopted community eligibility, to 75 percent in 2017, because the majority of students in the state now attend schools that provide schoolwide free meals—and all the students in those schools were counted as low-income on the NAEP.
Many of the students newly counted as low-income because of schoolwide eligibility had higher test scores than students who were already considered low-income, the Urban Institute found. This meant a group of students with higher scores than the average low-income students and lower scores than the average higher-income students switched categories, artificially raising the performance of both groups.
The new index may provide a much-needed new model for measuring child poverty in states, said Rachel Anderson, the vice president of the Data Quality Campaign, which works with states on collecting and using longitudinal student data, including socioeconomic status. Some still use free and reduced-price lunch status, though a majority of states try to directly certify students’ income through an array of procedures. NAEP’s index will allow educators to compare how students of different income levels are performing across different states.
“Individual descriptors like income are helpful, but they don’t tell the full picture,” Anderson said, noting that less direct indicators—like neighborhood access to transportation and healthcare—can help educators and policymakers pinpoint which resources are most important to student success.
Linking resources to achievement
These more complex data may need more updating, however. For example, while the number of print books in the home has remained closely linked to family income over the last several decades, it is not clear how that will change as more children grow up reading primarily digital texts.
“It’s really tricky, right? Because what if you have a subscription to a site that provides you an unlimited number of free books, what would it mean?” Broer said.
“In general, there has been a decline in reported [print] books at home, but still students who report books at home, have on average a much higher performance than students who have less books at home—it’s actually one of the strongest variables related to performance,” Broer said. “I would categorize it as something related to social or cultural capital.”
Creating a more complex picture of poverty can help educators understand what supports students most need, Anderson said.
“I would like to see this federal effort prompting different kinds of conversations at the state level,” Anderson said. “It would be great to see states taking a cue here and thinking about more sophisticated ways that they can understand their students’ needs and their ability to address those needs.”
