When education researchers want to measure the collective poverty level in a school, they typically use the same yardstick: the percentage of students who qualify for free or reduced-rate meals under the federal school lunch program.
But dissatisfaction with that indicator is prompting some researchers to cast about for better ways to gauge the socioeconomic status of schools. While those efforts are still in the preliminary stages, they reflect a broader debate about the adequacy of federal poverty indicators and could, in the long term, influence a wide range of education policy decisions affecting poor children.
“There are problems with the free- and reduced-price-lunch measure, and if it’s not accurate, we need to fix it,” said Mark S. Schneider, the commissioner of the U.S. Department of Education’s National Center for Education Statistics, which is set to begin testing alternatives to the measure next year.
Education researchers have relied on the subsidized-meals indicator almost since the National School Lunch Program began in 1946. The program, run by the U.S. Department of Agriculture, feeds more than 26 million children in nearly 100,000 public and private U.S. schools.
While any child can buy a lunch under the program, free meals go to children whose family incomes fall at or below 130 percent of the federal poverty level; reduced-rate meals are for those whose families earn no more than 185 percent of that level. Schools that participate must keep track of who gets what kind of subsidy.
The statistics from the program are used for a wide range of purposes. Education researchers and federal testing programs use them ubiquitously as an indicator of school-level poverty, and districts use them to distribute money from the federal Title I program for disadvantaged children.
“The reason people use the free- and reduced-price-lunch indicator is because it’s handy, it’s there, and people collect it every year,” said Gary Burtless, an economist at the Brookings Institution, a Washington think tank.
Yet most scholars agree that the measure is also not as accurate as they would like. In fact, outside of education—in disciplines such as sociology, for example—it’s rarely used.
One concern is that educators and local officials have little incentive to verify program eligibility. Another is that students tend to drop out of the program as they grow older, often because they don’t want their peers to know their families’ economic status.
Statistics published by the USDA in 1996, the most recent available, showed that across the schools that provided subsidized lunches, program-participation levels fell from 69 percent of 1st and 2nd graders to 35 percent of students in the 11th and 12th grades.
‘Different Kind of Poverty’
To reduce the perceived stigma of getting subsidized meals and to ensure that children are well fed, some schools in high-poverty areas serve them to all students. That well intentioned practice can further muddy the measure’s accuracy for researchers.
“We understand the school-meals-program data is the best information available about the income of families at the school level,” said Susan Acker, a spokeswoman for the USDA’s Food and Nutrition Service. “But this was not specifically designed to provide a statewide or districtwide measure of poverty.”
Researchers and federal statisticians are experimenting with alternative ways to measure the level of poverty in a school or the neighborhoods surrounding it. The search for alternatives stems from concerns about the accuracy of using student eligibility for the federal subsidized-lunch program as a proxy for school poverty.
“Geocoding”—Assigning geographic identifiers to students so that analysts can gather income information from the U.S. Census Bureau about the areas in which the children live.
The Dissimilarity Index—A method that uses Census data to calculate the proportion of poor families that would have to move out of the neighborhood surrounding a school in order to achieve an even socioeconomic distribution.
The Isolation Index—A calculation, also drawn from Census data, that measures the extent to which poor people in a school’s neighborhood are likely to be in contact with members of the same socioeconomic group.
Home Resources—Survey questions asking students about the kinds of learning resources available in their homes, such as the number of books or magazines, encyclopedias, or computers.
The measure is also imperfect because the eligibility span is so wide, fueling some analysts’ view that it fails to capture the full dimensions of poverty in many neighborhoods, especially those with concentrated poverty.
“Concentrated poverty is really a different kind of poverty,” said Anja Kurki, an analyst for the American Institutes for Research, a Washington-based research organization. “The whole idea is that in these neighborhoods there might not be role models, and there are high concentrations of single-female-headed families, and parents have to go to other neighborhoods to find work.”
Mr. Kurki led a team of AIR researchers who last year tested alternatives to using the subsidized-lunch indicator. They wanted to see how closely the poverty levels and predictions of children’s academic achievement that they yielded corresponded with those produced by the meals measure. They studied four alternatives, all of which rely on U.S. Census Bureau data to estimate poverty in a school’s attendance area.
One indicator, for instance, calculates the proportion of poor families in school neighborhoods; another figures the percentage of single-parent families.
The researchers also tested an “isolation index,” which gauges the extent to which poor people are likely to encounter members of the same social group, and a “dissimilarity index,” intended to describe the proportion of poor families who would have to move out of the neighborhood to even out the socioeconomic distribution.
All the measures, except for the dissimilarity index, tracked relatively closely with the subsidized-meals indicator, the study found. Three of the four—the dissimilarity index was again the exception—predicted student achievement better than the meals measure.
But because the alternatives draw on Census data collected once a decade, the indicators grow less reliable as neighborhoods change over time. They also fail to account for the growing numbers of children in magnet, choice, or charter schools outside their attendance zones.
“I think these kinds of measures are useful, and they tell us something else beyond individual levels of poverty, but they are not by any means perfect,” said Ms. Kurki. She also cautioned against reading too much into the findings, because the study is based on data from just seven urban districts.
NAEP Test Planned
Mr. Schneider, the NCES commissioner, plans to embed a much larger test of alternative measures next year in the National Assessment of Educational Progress, a set of congressionally mandated tests taken by nationally representative samples of students every few years.
The NAEP program is due to test students in mathematics, reading, and writing next year at grade levels that vary by subject.
The NCES, which administers the national assessment, has relied for years on subsidized-lunch statistics to get a handle on children’s economic status, as federal law prohibits the tests from including questions on how much money test-takers’ parents earn.
But next year, Mr. Schneider said, the statistics center will experiment with some survey questions that would be appropriate for test-takers of different ages that might yield the same, or even more reliable, information on their socioeconomic circumstances at home. Finding the right questions—ones that will yield accurate information from younger children, for example—is difficult, Mr. Scheider said.
“If you ask 4th graders, ‘Do your parents have a law degree or a Ph.D. or maybe never graduated from college?,’ they’re not likely to know,” he said.
“And you can’t ask if there’s a set of encyclopedias in the house because, well, nobody has encyclopedias anymore,” he added, referring to a survey question that was once thought to be a reliable indicator of family socioeconomic status. “Maybe we’ll have to ask about computers or high-speed Internet,” he added.
The statistics agency has not yet determined what questions it will pose, Mr. Schneider said, because the survey items are still being piloted.
The commissioner said the center would also draw on federal Census data to “geocode” student test-takers and gather information on the neighborhoods where they live. The idea, he said, is to better account for children taking advantage of school choice options.
“Around 75 percent of kids are still going to their neighborhood school,” he said. “For those kids who aren’t, this will provide a much tighter match.”
To keep the data fresh, Mr. Schneider added, the NCES may tap into the American Community Survey, a study launched last year by the Census Bureau that surveys adults in smaller geographic tracts every three to five years, rather than every decade.
The statistics agency ultimately plans to examine the results from the different methodologies and see how closely they correspond to figures on free- and reduced-price lunch and other data.
Mr. Schneider said the experiment’s timing is particularly opportune because the NCES will be able to cross-check the results from the survey questions and Census data against detailed family-income data from another massive Education Department study, the Early Childhood Longitudinal Study, Kindergarten Cohort Class of 1998-99, which has been tracking 22,000 students since kindergarten.
Students in that study are now in 8th grade, and researchers will be in the field collecting data on them at the same time that the NCES will be administering NAEP reading, writing, and math tests to 8th graders. By ensuring that those students are included in the NAEP tests, researchers will have a rare opportunity to gauge the accuracy of the new methods for that age group.
“The whole idea is to get some idea of the richness of assets in the home, to get an indicator of the kind of support parents give their kids,” the NCES chief said.
Reliable Alternatives
Mr. Schneider said he expects to be able to report the results of the NAEP trials within a year. But if any of the alternatives turns out to be a reliable gauge of poverty, it’s an open question whether districts will want to take the next step and use a different measure in deciding how to give out Title I money.
“It would be possible to talk about using other kinds of measures, but they would have to be reliable, not subject to manipulation, and fairly contemporary,” said Phyllis McClure, a Washington-based Title I consultant. “What I mean by contemporary is, what if a housing project closed down and the population of poor students declined very rapidly? You would not want to be allocating funds to schools when the kids are no longer there.”
The experiments with alternatives to the subsidized-meals indicator come amid debates among policy experts over the accuracy of the various measures the federal government uses to delineate who is poor and who is not. Currently, federal poverty guidelines set the threshold at around $20,000 a year for a family of four.
“Anything you do on the threshold or the research side is going to have an impact on the numbers of people who are considered to be living in poverty,” Mr. Burtless said. “But we need rough and ready measures of the resources that families have.”
