Study Provides Data Showing Ill Effects of 'High Stakes' Tests
Offering statistical evidence for what some educators have long suspected, a new study suggests that some "high stakes'' student-testing programs are pressuring schools to either retain low-achieving students in grade or put them in special-education programs.
The federally funded study highlights growing concerns among a number of regular and special educators over the unintended effects some educational reforms have on disabled and low-achieving students across the nation.
Researchers at the State University of New York at Albany studied 12 elementary schools in New York State over a decade. After the state began using school "report cards'' to disclose to the public individual schools' scores on a statewide 3rd-grade reading test, the researchers found, the number of students identified as disabled increased in every school.
Moreover, it found, students were increasingly being retained at earlier grades or were placed in transitional programs designed to give low-achieving kindergartners an extra year to mature.
The researchers said such practices effectively removed large numbers of students from the assessment stream or delayed their participation in the tests. If the scores of many of those students had been included in the school report cards along with those of other pupils their age, the researchers calculated, they would have lowered the schools' reading-achievement scores and wiped out most of the gains reported by those schools.
"It appears unlikely that significantly increasing the incidence of early grade retention and special-education enrollments were intended effects of the various educational-reform initiatives undertaken in New York State,'' write the study's authors, Richard L. Allington and Anne McGill-Franzen. "Yet these are the undeniable outcomes of this sample of schools.''
Raising the Stakes
The study's data are reinforcing longtime concerns of some educators.
As early as 1988, the U.S. Education Department, in its annual report to Congress on special education, questioned whether the rising special-education enrollments occurring nationwide throughout the 1980's might be due to trends among states toward raising academic standards and instituting high-stakes assessment programs.
Special educators worry that schools have begun to overidentify the number of students in need of special education.
"There have been a lot of expressions of opinion on that point, but this [new study] is the first hard data I've seen showing what's actually happening to kids,'' said James E. Ysseldyke, the director of the National Center on Educational Outcomes at the University of Minnesota.
"Any time you hold districts accountable in a comparative sense, you can expect there's a big incentive to keep kids who people think are going to do poorly out of the picture,'' he added.
According to a 1990 study, 47 states require schools to assess students at some point. And 39 of those states, including New York, mandate the use of state-developed, state-selected, or state-approved tests to carry out the assessments.
There is widespread variation, however, in the degree to which disabled students' scores are included in publicly disclosed test results. (See Education Week, April 29, 1992.)
Mr. Ysseldyke said some states automatically exempt special-education pupils from taking the tests or exclude their scores from reports, while others allow schools to make those decisions on a case-by-case basis. Some states that include disabled students' scores in reports may note the percentage of such students taking the tests in a given school.
New York State has had a statewide pupil-evaluation program in place since 1965. In 1984, however, as part of a far-reaching curriculum-reform effort, the state began issuing widely publicized report cards on student achievement in schools and districts. The state education department also began compiling and publishing lists of the "worst schools'' in the state that were based on scores from the statewide testing program.
Disabled students in the state are permitted to take the tests, but their scores are not included in the accountability profiles issued by the state.
For the purposes of their study, which was published last month in the journal Educational Policy, Mr. Allington and Ms. McGill-Franzen divided the 12 elementary schools into two groups: those with relatively stable scores over time on the 3rd-grade reading test and those with improving reading scores.
Across all 12 schools and grade levels, the researchers found, the proportion of students annually identified as disabled increased from 2.1 percent during the 1978-79 school year to 3.5 percent in 1988-89.
The increase was greatest, however, in schools that reported improving test scores.
Moreover, in the schools that started out with low scores and improved over time, the study found, the increase in the rate of children being classified as disabled was three times greater than it was in schools with historically high achievement scores.
"I don't think there's a shred of evidence that there are more kids with more problems,'' said Mr. Allington, who is also a professor of education.
Mr. Allington and Ms. McGill-Franzen said some school administrators acknowledged that increased pressure from the testing program may have contributed to the rise in special-education enrollments.
"School principals would tell us they stayed up through the night on the eve of reporting the test scores to see what children were in the batch they could pull out,'' said Ms. McGill-Franzen, who is also an assistant professor of education at SUNY.
"Superintendents have been known to walk the corridors of a certain school and tell 3rd-grade teachers, 'You can run but you can't hide,' '' she added.
"I don't think principals are saying, 'I need to wash my hands of this,' '' Ms. McGill-Franzen continued. "I think they're trying to juggle different kinds of pressures and serve the needs of the kids as well.''
In contrast to the rates at which pupils were being identified as disabled, retention rates remained stable over all in the schools studied. Children were, however, being increasingly held back a year in the primary grades--usually before the 3rd-grade reading tests.
Much of the increase in retention, the researchers said, was due to increasing use of transitional 1st grades for students not deemed ready to enter a regular 1st-grade program. In one school, nearly half of the original kindergarten class was placed in such a program.
Again, as was the case with special education, the researchers found the rate of increase in the incidence of early-grade retention to be greatest in schools with improving scores on the reading-achievement tests.
Artificially High Scores?
To find out whether including the scores of all those students retained in grade or classified as disabled would have made a difference in scores on the reading-achievement tests, the researchers more closely examined seven of the 12 schools.
For these students, the researchers tracked down their reading-achievement score on whatever standardized test they took at the end of their fourth school year. They then adjusted those scores for comparability. The schools' scores on the tests were then recalculated using all of the previously excluded scores.
"In virtually every school reporting substantially improved student achievement,'' Mr. Allington said, "the whole of the achievement could be accounted for by these practices.''
"One thing seems obvious: The 'snapshots' of school achievement presented in state accountability profiles such as those used in New York are far more subject to interpretation than has typically been acknowledged,'' the researchers concluded in an earlier report on their study.
But a spokesman for the state education department said he disagreed with some of the researchers' conclusions.
"I don't see [the state testing program] as an incentive'' for excluding or retaining children, said Larry Waite, the director of program development and support services in the department's special-education division. "Hopefully, we're looking at multiple factors in combination as to why students are retained or placed in special education.''
Statewide, the percentage of students identified as disabled increased from 6 percent in 1978 to 10 percent in 1991, according to the study. Mr. Waite said that increase, however, reflected an increase in special-education enrollment taking place nationwide at the same time.
Some of the increase, Mr. Waite said, is attributable to the fact that New York, like other states, began to identify children in need of services before they reached school age. He also noted that the proportion of students in New York schools who are identified as disabled is still below the national average of about 12 percent.
Experts agree that, while not all disabled students should be forced to take a standardized test, a great many are able to take the tests.
And a handful of states are beginning to re-examine their testing practices with an eye toward finding a systematic and equitable way to include disabled children in the tests, said Margaret McLaughlin, the director of the Center for Policy Options at the University of Maryland.
Ms. McLaughlin said that some school administrators in Kentucky, for example, found that disabled students helped boost their schools' scores on statewide tests when the state replaced its standardized paper-and-pencil tests with a performance-based testing system.
"At some point, the state or someone has to come in and hold schools' feet to the fire,'' she said. "If a school has exempted 25 percent of the population, we need to find out why.''
"Some educators say to us, 'You guys have [individualized education plans], so you have accountability built in already. What's the big deal?' '' Ms. McLaughlin said.
"It is a big deal if you're not accounting for all of your