The “anomaly” that produced a bafflingly steep drop in some scores on the National Assessment of Educational Progress’s 1986 reading test stemmed from a technical flaw in naep’s comparative analysis, officials said last week in their report on the test.
The report--delayed by more than five months while researchers investigated why the results seemed to indicate a huge drop in reading performance among 9- and 17-year-olds between 1984 and 1986--reflects the problem caused by the anomaly, naep officials acknowledged.
Rather than compare performance over time, the report examines only factors that relate to students’ reading achievement on the 1986 test, which tested the reading skills of 36,000 3rd, 7th, and 11th graders. naep officials had planned to publish a study showing trends in students’ reading performance since 1971.
A host of testing experts failed to find the cause of the anomalous data, and naep officials last week released a separate report that describes their investigation.
But Archie E. Lapointe, director of naep, said the problem was most likely in the statistical technique used to compare the results with those of the previous assessment, rather than in the results themselves.
“Where we think we have the problem is in [the comparative] procedure,” he said. “We are not comfortable yet that the [comparative] results we have come up with are right.”
He said the agency was confident, however, that the data were sufficiently valid to support the study released last week, which represents a “snapshot” of how students performed.
“All you need to do that is a good, solid, valid, reliable reading test,” he said. “We are convinced this is a good, solid, valid, reliable reading test. Every expert we consulted agrees with that.”
Chester E. Finn Jr., the U.S. Education Department’s assistant secretary for educational research and improvement, said that he was aware of the experts’ testimony and was “willing to take their word.”
“It is entirely possible the test can be valid unto itself and at the same time not comparable to something else,” he said.
But he added that the department was “disappointed” that naep was unable to provide data on reading trends.
“This is not the report we would have liked to have had,” Mr. Finn said. “It’s what textured soy protein is to beefsteak: nourishing, but not very enjoyable.”
Some Still Lagging
The report, “Who Reads Best? Factors Related to Reading Achievement in Grades 3, 7, and 11,” found that, as in previous naep studies, the performance of black and Hispanic students lagged behind that of their white counterparts.
At the 11th-grade level, it found, average scores for black and Hispanic students, as well as for those from disadvantaged urban communities, are only slightly above the 7th-grade level for students nationwide.
In addition, it found that students’ scores tended to increase as they became older, but that the difference between the low-performing and high-performing students was about the same at each grade level. This finding “suggests that American schools continue to have difficulty narrowing the gap between better and poorer readers as they progress through school,” the report states.
The report also analyzed the relationship between instructional techniques and performance. Reading research, it says, indicates that, to be effective, instruction should focus on comprehension before, during, and after students read.
Instruction Seen Varying
For example, it notes, teachers can preview the material and point out new vocabulary, give out lists of questions for students to consider2p4while reading, and require students to think about, discuss, and support their opinions about the text.
Overall, the report states, about two-thirds of 3rd graders, and more than three-fourths of 7th and 11th graders, reported that their teachers used these techniques.
However, it adds, poorer-performing students “receive qualitatively different instruction from their higher-performing classmates.”
Better achievers, it found, reported engaging in more before- and after-reading activities, and were “more likely to engage in thought-stretching activities” than their lower-performing classmates.
The poorer students, by contrast,generally received more help getting through the text.
“While this help is useful,” the report states, “it may be unnecessarily limiting--keeping students from also beginning to practice the very kinds of skills and strategies that are used by their higher-performing classmates.”
“Classroom practices are related to performance,” said Mr. Lapointe. “Poor readers don’t seem to be exposed to the variety of reading techniques that good readers are.”
The 1986 study also found, as have previous assessments, that most students have difficulty thinking about what they have read, and expressing their ideas in writing.
“Reading instruction at all levels must be restructured to ensure that students learn to reason more effectively about what they have read,” the report states.
Out-of-School Reading
In addition, the study found that poorer readers tend to read less often in school than better readers, but that the two groups read nearly the same amount outside of school.
“This suggests that if these students are exposed to a richer fare in school, that might be helpful,” said Mr. Lapointe.
The report notes, for example, that 47.6 percent of poorer readers said they read for fun daily, compared with 56.2 percent of the top readers. But three-fourths of the top readers said they read in school every day, while only 56.6 percent of the poorer readers said they did.
“These results reflect a dilemma,” the report states. “The poorer readers presumably have more difficulty reading on their own and, therefore, are less likely to be encouraged to do so. Yet the fewer opportunities they have to read, the fewer are their chances to become better readers.”
“It may be that expectations for the poorer readers are sometimes set too low, asking them to read less than they are capable of doing,” it concludes.
Copies of the report are available for $12.50 each from the National Assessment of Educational Progress, CN 6710, Princeton, N.J. 08541-6710.
