Pupil Gains Not Just Mythical, Test Publisher Says Study Shows
A leading test publisher, aiming to refute charges that standardized-test scores are inflated, has found in a new study that elementary-school students registered substantial gains in basic-skills achievement over the past decade.
The study by ctb/McGraw-Hill found that, in 1987, students in grades 1 to 8 who took its Comprehensive Test of Basic Skills scored an average of 6.63 percentile points higher in reading, 10.83 points higher in language, and 14 points higher in mathematics than a 1981 comparison group.
On the California Achievement Test, it found, between 1978 and 1985, scores increased by an average of 10.1 percentile points in reading and 10.78 points in math.
The findings help explain why separate studies have shown that most elementary students score "above average" on the nationally normed tests, according to Paul L. Williams, director of research and measurement for ctb/McGraw-Hill.
On nationally normed tests,t scores are compared with others, which can be up to seven years old, achieved by students in a norm group.
"The fact is that the gains are real, and that national averages have increased in the past 10 years," said Mr. Williams.
Moreover, he noted, the gains were greatest among those who scored at or below national averages a decade ago. Such increases, added David Deffley, general manager of the Monterey, Calif.-based firm, demonstrate the success of federal compensatory-education and state basic-skills programs.
In addition, noted Mr. Williams, the tests themselves have helped improve instruction by enabling teachers to identify students' weaknesses and target resources to needed areas.
The new study represents the testing firm's latest response to critics who have charged that the test scores are misleading.
A controversial study last year by Friends for Education, a West Virginia advocacy group, found that the overwhelming majority of elementary students scored above national averages on nationally normed tests. This came about in part, the group charged, because test publishers and administrators manipulate the scores to make student achievement appear higher8than it actually is.
But the ctb study found that achievement has in fact improved, and that student scores should be higher than they were when the averages were set.
"Achievement has gone up, and the standard is tougher now," said Mr. Williams. "It's harder to be at the 50th percentile."
The study compared the scores from a sample of students who took the ctbs in 1987 with those in a national sample from 1981. Itthat the average overall gain per grade ranged from 8 points at grade 8 to 13.3 points at grade 2.
The largest increase, it found, was in 2nd-grade mathematics, where the nation as a whole achieved at a much higher level. A student who scored as well as or better than 68 percent of all students in 1981, Mr. Williams noted, would have scored as well as or better than 50 percent of all students in 1987.
A separate study of cat scores, which compared national samples of students in 1977 and 1985, found similar gains. Students in grade 9, it found, gained an average of 11.5 national percentile points, while 6th graders gained an average of 10.17 points, and 3rd graders gained an average of 9.67 points.
Researchers familiar with the analysis questioned last week el10lwhether it proved that the gains in student scores reflected actual gains in achievement.
Daniel Koretz, senior social scientist for the rand Corporation, pointed to the fact that the 1987 scores were for students who had taken the ctbs, rather than a random sample. The test group's increases could reflect teachers' efforts to boost scores by "teaching to the test,'' he said.
Moreover, the ctb study did not address student achievement in higher-order skills. Other national studies, such as the National Assessment of Educational Progress, have also found gains in basic-skills achievement since4the late 1970's.
But naep studies have also found that few students can perform more complex problem-solving tasks, which many argue are necessary for college work and the job market.
Mr. Williams said the firm was unable to discern the level of students' abilities in higher-order skills. But he contended that the improvements in basic skills are significant signals that growing numbers of children have a foundation for more advanced work.
"If children don't have basic skills," he said, "there is no way they can do higher-order skills."