But determining how to accommodate many such students raises many practical and financial questions.
Between 40 percent and 50 percent of all students with disabilities are excluded from the sampling procedures used in some of the most prominent surveys and assessments used to measure the nation’s educational well-being, according to a new study.
Researchers from the National Center for Educational Outcomes at the University of Minnesota, which conducted the study, said the findings raise questions about the reliability of such well-known measures as the National Assessment of Educational Progress and the National Educational Longitudinal Study. By excluding some of the students who have the most trouble learning, the researchers said, such estimates could be painting an overly optimistic picture of educational progress.
The findings come as national efforts to improve education are focusing to a great degree on such measures. The National Educational Goals Panel, for example, has begun producing annual reports on how well the nation and individual states are doing in meeting the six national education goals. And federal lawmakers are debating whether to establish a national assessment system for measuring student achievement.
“We really worry about the fact that, as all this national excitement about goals is occurring, it’s easy to be forgotten if you’re not part of the picture,’' said Martha L. Thurlow, the assistant director of the center.
Treated as ‘Outliers’
The study’s findings were presented during a meeting here last week of national special-education organizations and state directors of special education. They mirror the findings of an earlier study by the center that focused on the degree to which special-education students participate in state-assessment practices. (See Education Week, March 4, 1992.)
The researchers looked at the testing policies and procedures of nine national surveys or tests often used to guide education policy. In addition to NAEP and NELS, they included the Longitudinal Study of American Youth, the National Adult Literacy Survey, and several population and health surveys conducted by the Department of Commerce or the Department of Health and Human Services.
The researchers said disabled students are left out of many of those study samples in a number of ways.
In telephone surveys, for example, children who live in families too poor to afford a telephone would not be included. Researchers believe disabled students may be disproportionately represented in such households.
Research programs that seek to test students often automatically exclude about 7 percent of special-education students because they focus only on regular schools, according to the study. The researchers said 315,000 disabled students are taught in special settings or home-based instructional programs.
Disabled students are also omitted from testing samples, the researchers said, when test-givers draw students for the tests from school grade rosters. Many students with disabilities are in ungraded school programs.
Some of the testing programs have written policies to guide local school officials on whether to include students with disabilities. They may allow, for example, the exclusion of pupils with physical disabilities or those who do not take mainstream classes.
In practice, however, the study says, those policies are often loosely implemented by local school officials. Despite written guidelines for the NAEP program, for example, researchers found that the rates at which states were excluding disabled students from the 1990 state-level NAEP exams ranged from a low of 33 percent in Minnesota to 87 percent in the District of Columbia.
Local school officials may decide to exclude students for a number of reasons--many of which are understandable, the researchers said. Some students, for example, clearly cannot complete an hour-long exam.
The researchers said some students, such as those who are blind, are also being excluded because they need extra help in completing an exam.
“Those students judged by teachers to be ‘at risk’ for experiencing discomfort during the testing situation were also excluded,’' the report states.
And, in some cases, the researchers surmised that school officials have left out such students for fear they would lower the scores.
The researchers said many such children could easily take standardized tests and many already do.
“Not only does the treatment of students with disabilities as ‘outliers’ in national and state data-collection programs make it difficult to produce accurate national and state statistical estimates for this population,’' the report concludes, “it also raises questions about bias being present’’ in many such estimates.
Remedial Efforts
Spokesmen for several of the national testing programs cited in the study acknowledged that the exclusion of disabled students was a weakness in their programs.
In the NAEP program, for example, local school officials are permitted to exclude non-English-speaking students and those who have individualized education programs and are mainstreamed less than half the time. In deciding whether to include others, local officials are encouraged to “err on the side of inclusion,’' according to program guidelines.
“Some people make those decisions liberally and others may make them conservatively,’' said Ramsay Selden, the director of the state education-assessment center of the Council of Chief State School Officers, which is helping to develop new NAEP exams. “You don’t have a full representation of all the kids serviced by a school system of a state or the country as a whole.’'
Both NAEP and NELS, recognizing the problem, have begun collecting information on excluded students with an eye toward taking them into account. Their data, however, show that only about 5 percent of the sample population was excluded.
The National Center for Education Statistics, which funds those testing programs, is looking into the issue.
But determining how to accommodate many such students raises many practical and financial questions.
“I think it’s a very important issue,’' said Thomas Hoffer, the associate director of the Longitudinal Study of American Youth, sponsored by the National Science Foundation, “but any given study can’t do everything.’'
