A study of state trends on the National Assessment of Educational Progress from 1990 to 2000 underscores how challenging it may be for states to close the achievement gap between minority and nonminority students, despite a federal mandate that they do so within 12 years.
The study, “Tracking the Improvement in State Achievement Using NAEP Data,” by David Grissmer and Ann Flanagan, two researchers at the Santa Monica, Calif.- based RAND Corp., was presented at a meeting in Washington last month.
The researchers found almost no state made significant gains on NAEP reading tests during those years. In mathematics, about half the 23 states for which data were available made some progress in closing the achievement gap between black and white students, the authors found, but those gains were “very small compared to the state black-white gap.”
“Even if such trends would continue for these states,” the authors predict, “it would take decades to close the gaps in these states. And many states show widening gaps, although the amount of widening is also small compared to current gaps.”
Exclusion Rates
The study also found that as much as 30 percent of the improvement in state math scores between 1990 and 2000 could be attributed to changes in the proportion of students who were excluded from taking NAEP in various states. Students were barred from taking the assessments either because they spoke limited English or because they had disabilities that prevented them from participating.
Since 1998, those exclusion rates have been rising, and the pattern of such rates across states has changed. One reason is that, until recently, NAEP did not test disabled students who either did not take their state-administered tests or who took the state tests with accommodations, such as extra time. As states have moved to accommodate students taking their tests, as required by federal law, their exclusion rates on NAEP have increased.
Nationally, between 1 percent and 2 percent of students have been shut out from taking the tests because they have limited fluency in English, and between 4 percent and 6 percent have been excluded because they have disabilities.
Partly because of such concerns, the governing board that oversees the federal testing program has decided to permit students who need accommodations on their state tests to take NAEP with such assistance, as long as it does not change the nature of what’s being measured. (For example, students cannot have NAEP reading tests read to them.) In the future, NAEP will include the performance of such students in reporting state scores.
Mr. Grissmer and Ms. Flanagan noted that no current evidence exists that higher exclusion rates are “conspiratorial,” but rather reflect the gap until now between NAEP testing practices and increasing attempts to offer accommodations at the state level.
The researchers concluded that a new level of sophistication is going to be required in the reporting and interpretation of NAEP scores, given the increasing importance of NAEP. Under the “No Child Left Behind” Act of 2001, a sample of 4th and 8th graders in every state will be required to participate in NAEP reading and math tests every other year, in part, to verify trends on state assessments.
In addition to adjusting NAEP scores to better reflect exclusion rates, the researchers suggested, changes in the sample sizes may be needed across states to reflect more accurately shifts in performance over short time periods and shifts in gap closing.
Pat J. DeVito, the director of the board on testing and assessment at the National Research Council, said the study “did a good job based on the information that it had.” But he cautioned that “we’re going to need more sensitive methodology to be able to determine the gaps” than NAEP currently provides.
He noted, for example, that NAEP did not include large enough sample sizes in many states to measure the closing of achievement gaps and that, even when NAEP shows progress, the survey offered no specific information on state education reforms that could suggest why such progress has occurred.
