Federal statisticians are reviewing scores from a 1998 national reading assessment to see if some states improved their standing because they excluded higher proportions of students with disabilities and limited English skills than they did in earlier tests.
States that registered some of the largest gains on the 4th grade test also increased the rate at which they removed students from the testing sample because of their disabilities or their inability to understand English.
A preliminary review of procedures found that states carefully followed the rules established for the 1998 National Assessment of Educational Progress testing, according to Pascal D. Forgione Jr., the federal commissioner of education statistics. The next step will be to calculate whether the exclusion patterns boosted or dragged down states’ scores.
“Whatever took place has to be systemic,” Mr. Forgione said last week. “It wasn’t that anyone was trying to influence NAEP scores. We found nothing irregular at all.”
Focus on Conn., Ky.
In the national sample for the 1998 reading test, 11 percent of potential test-takers were excluded, either for their disability or lack of English skills. The percentage of disabled students excluded from the assessment stayed at 6 percent in 1992, 1994, and 1998. School officials removed 5 percent of the sample because of language barriers in 1998, up from 3 percent in both 1992 and 1994.
But in some states, the exclusion of children with disabilities or limited English proficiency skyrocketed. Kentucky pulled out 10 percent of the students selected for its sample, compared with 4 percent in both 1992 and 1994. Connecticut removed 12 percent of the students selected to participate, compared with 8 percent in 1994 and 7 percent in 1992. In both states, the growth in exclusions was tied to students with disabilities, contrary to the national trend.
Connecticut and Kentucky were two of three states that scored statistically significant increases over 1992 and 1994. (“States Committed to Standards Reforms Reap NAEP Gains,” March 10, 1998.) The other, Colorado, removed about the same ratio of students all three years.
The review by the National Center for Education Statistics, a branch of the Department of Education, will determine whether the increase in students held out of the test helped states such as Connecticut and Kentucky. It will also examine whether other states’ scores were dragged down because they included more students with disabilities or English deficiencies.
The theory is that a state would raise its overall scores by excluding high portions of disabled and LEP students from participating in NAEP, a federally sponsored sampling of students in core subjects.
One researcher who compared exclusion rates with states’ increases saw evidence that the theory might be true. “There might be something there that’s worth looking at further,” said Richard S. Brown, a project director for the National Center for Research on Evaluation, Standards, and Student Testing, a federally funded project based at the University of California, Los Angeles.
No Accommodations
Throughout its history, NAEP has not offered accommodations such as extended time or proctors to read questions for students with special needs. Many states, including Kentucky, routinely offer such assistance on their exams.
For the 1998 exam, the National Assessment Governing Board, which oversees NAEP, issued new rules. Under those rules, any child with an individualized education plan--which is required by federal special education law--that explicitly says he or she needs accommodations to be tested fairly is supposed to be removed from the sample.
In contrast, when the reading exam was given in 1992 and 1994, students were to be included in the NAEP testing if they spent at least half their time in a mainstreamed classroom. Principals also could pull students from the test.
In Kentucky, pressures to include disabled students in statewide testing have led to a rash of IEPs detailing what assistance those students should get, said Richard G. Innes, a citizen activist from northern Kentucky, who tracks state education trends.
Jim Parks, a spokesman for the Kentucky education department, acknowledged that schools felt that pressure. But he said that the disabled students removed from NAEP wouldn’t have depressed the state’s score, had they been included.
In Connecticut, the increase in exclusions of students with disabilities simply raised the number to match the special education rate in the state, according to Thomas W. Murphy, a spokesman for that state’s education department.
