Study Disputes Link Between Reforms, Rise in Test Scores
Washington--The recent rise in the scores of students on standardized tests is not attributable to this decade's wave of school reforms, a new study paid for by the Congress argues.
The upswing in standardized-test scores not only predates education reforms of the 1980's, but "might well have continued in their absence," states the report, prepared for the Senate Education, Arts and Humanities Subcommittee by the Congressional Budget Office.
The report disputes common assumptions about why scores fell in the 1960's and began rising in the mid-1970's--such as that students watched too much television, or that high-school graduation standards declined.
And it contends that, in any event, test-score trends are an inadequate yardstick for measuring reform, which it says hinges on a far more complex range of factors than the tests measure.
A more complete portrait of achievement trends could be painted, it suggests, by juxtaposing a wide array of data weighing academic, demographic, and societal factors.
Daniel M. Koretz, the cbo's analyst for elementary and secondary education, wrote the 104-page analysis, the second of two studies on testing by the budget office.
"Many people have used trends in test scores and assumptions about their causes not only to formulate new education policies, but also as a basis for presuming their effectiveness," the report states.
But, the cbo asserts, "the results of this analysis suggest that the effectiveness of the current wave of initiatives should not be presumed on the basis of assumptions about what caused past trends."
Public awareness that student achievement seemed to slip sharply in the 1960's and 70's, lagging also in international comparisons, has helped drive a wide range of reforms, including higher pay and stricter certification standards for teachers to expanded testing and more rigorous requirements for students.
While the cbo report does not disparage policies intended to counter declining test scores and the factors presumed to have causedthem, it cautions that "assuming greater consistency than actually exists" between the trends and the initiatives can misdirect policymaking and obscure the importance of other contributing factors.
The report says demographic and cultural shifts, rather than stiffer graduation standards and other hallmarks of reform, have influenced test-score trends.
But, it cautions, the mix of other factors that fueled achievement fluctuations remains largely unknown.
The cbo analysis found, for example, that changes in the size of students' families and in the ethnic mix of test takers made "particularly substantial contributions" to testing trends. Still, the report states, each of the two factors accounts for at most a fifth to a quarter of the test-score decline.
The growing percentage of minority students--who, on average, score lower on standardized tests--contributed to the decline and slowed the subsequent rise, according to the report; the changes in family size associated with the baby boom and bust, in the meantime, influenced shifts in both directions.
Factors that probably did not contribute significantly to the trends include many that "have gained widespread credence as possible causes," states the report, "Educational Achievement: Explanations4and Implications of Recent Trends."
The report--which expands on a 1986 cbo study that analyzed achievement patterns measured by several standardized tests (See Education Week, April 23, 1986)--says many observers, focusing mainly on the Scholastic Aptitude Test, link dips in scores to weak policies and relaxed requirements at the high-school level and attribute the rise in scores to the success of school reforms.
But in looking at a broad range of elementary- and secondary-school test data, the cbo found that the students whose test results triggered the upward trend in scores entered school in the late 1960's--a fact that links their performance to educational and social policies predating those of the reform era.
Reform Movement Credited
The report notes that state graduation standards did not change significantly between 1974 and 1979 and, thus, "appear not to have contributed directly" to the downturn intest scores during that period.
The cbo's findings cast doubt on claims by education officials, including Secretary of Education William Bennett, that the rise in test scores demonstrates the reform movement's success in recent years. Mr. Bennett in 1985 linked gains in sat score results to raised academic standards and declared the movement was holding its ground when the scores leveled in 1986.
Bruce Carnes, the undersecretary for planning, budget, and evaluation, said last week that the department has no "fundamental problems" with the cbo's study. The department "never ascribed all of test-score performance to a particular thing," but still sees education reform as a key player, he said.
"We don't think it's voodoo or sunspots," said Mr. Carnes, who cautioned against construing the study to mean that cultural and family characteristics outweigh human will and determination to succeed.
While those who link score trends and education reforms often assume a few key factors can be singled out and reversed to improve scores, the cbo study suggests that "changes in many, diverse educational factors might well be necessary to bring about increases in achievement as pervasive and large as the decline of the 1960's and 70's."
The report emphasizes that, despite the upward test trend in recent years, "the average level of perform8ance on some tests remains well below what many educators would consider acceptable."
'Plausible' Factors Cited
Citing factors that could be viewed as "plausible" causes of achievement trends, the report says a watering down of course content in secondary schools may have contributed to the decline in test scores. And it suggests that modest changes in the amount of homework assigned high-school students may have influenced the decline and rise of scores.
The report also states that the federal Chapter 1 program "could have contributed modestly" to the appreciable academic gains registered by black and Hispanic pupils relative to other students and that desegregation efforts may have bolstered the gains of blacks.
Further, changes in students' use of alcohol and other drugs may have figured in the decline and rise in scores, the report says, and it notes that reduced exposure to lead in the environment "might have contributed in small measure to the upturn."
But students' television-viewing habits cannot be linked to the trends because they have not changed substantially during the past two decades, the report states.
Without dismissing the relationship between teacher preparation and student achievement, the report also discounts the theory that the declining sat scores of prospective teachers depressed student achievement, since the decline did not begin until 1972.
The report contends that test-score patterns in some cases can mask or downplay the significance of education policies. For example, unusually rapid demographic changes or successful campaigns to lower the dropout rate may depress average scores "even if policies carried out during that time are beneficial," the cbo says.
Some potentially powerful factors whose influence on trends in test scores cannot be evaluated for lack of data, according to the report, include student attitudes and motivation, schools' emphasis on writing, and local graduation requirements.
The cbo adds that initiatives aimed primarily at high-school students may underrate the importance of elementary schooling, where critical skills are first taught.
Alternative Measures Urged
The report also says a growing tendency by teachers to "teach to the test," or gear instruction toward boosting test scores, could skew the significance of those scores. "Regardless of whether increased teaching to the test is desirable, it is likely to make trends in test scores a distorted proxy for achievement," it states.
The report urges educators to devise ways to address students' deficiencies underscored by the test data in "higher order" skills and to boost the performance of traditionally low-scoring groups. The report mentions recent policy proposals aimed at broadening the nation's portrait of student achievement, including the Education Department's proposal to expand the National Assessment of Educationel10lal Progress to allow state-by-state comparisons. (See Education Week, March 25, 1987.) Mr. Carnes termed that project "one of the most valuable tools" the Reagan Administration can contribute to education policy.
The report's analysis "argues strongly against relying solely on a single 'national achievement test"' for information about student performance. It would be "more reliable and informative," it argues, to use a number of tests.
"A comparison of several tests is often necessary to discern which results are consistent enough to provide a sound basis for policy," it states, "as evidenced by the several important instances in which the National Assessment of Educational Progress has yielded conclusions that are inconsistent with other data, and the wide variation in the results shown by other tests."
Copies of the cbo report may be obtained at no cost by writing the Publications Office, Congressional Budget Office, U.S. Congress, Washington, D.C. 20515.