Citing the potential abuses and misuses of tests, the Congress’s research arm last week raised caution flags in the current debate over creating a new national system of student assessments.
The Office of Technology Assessment, a nonpartisan analytical agency, expressed its concern at a hearing here on the recently released report of the National Council on Education Standards and Testing, which called for national standards for student performance and a system of assessments to gauge performance against the standards.
The concerns were contained in a study by the agency on educational testing, a summary of which was released last week.
Michael J. Feuer, the project director of the study, said the two-year analysis of testing in the United States, as well as a special study of national tests in other countries, found that “there is insufficient evidence to answer key questions regarding national testing.”
Among other issues, Mr. Feuer said, the Congress needs to clarify:
- The purposes for which such tests are to be put;
- The relationship between the tests and reform efforts already under way;
- Who would set the standards for the new tests; and
- Who will pay for the new assessments.
“Until these and other questions are addressed,” Mr. Feuer said, the “O.T.A. finds that placing a new test or system of tests into service at the national level could easily create new barriers to many educational reforms already under way and become the next object of concern and frustration within the American school system.”
Gov. Roy Romer of Colorado, the co-chairman of the standards council, said he was “appalled” at the suggestion, made by the O.T.A. and others who testified at the hearing, to delay implementing the panel’s report.
Many states, Governor Romer noted, are moving forward in developing standards and assessments, and are looking to the national level for guidance. He urged the Congress to adopt the council’s recommendation to create an entity to oversee the development of standards and assessments.
“We do not have the luxury of waiting that long,” Governor Romer said. “If we can go to the Persian Gulf and, in six weeks, conduct Desert Storm, we can develop a test in fewer than six years.”
Bella Rosenberg, a special assistant to Albert Shanker, the president of the American Federation of Teachers and a member of the standards council, added that a failure to act on the council’s recommendations could be harmful. Such a failure would keep in place the status quo, which is unacceptable, she said.
“This holds the promise of such a giant step forward,” Ms. Rosenberg said at the heating. “It is hard to believe there is so much attention on everything that could go wrong, rather than on how to do things right.”
‘High Stakes’ Uses
The hearing hero last week was the second held by the House subcommittee on elementary, secondary, and vocational education to examine the report by the standards council.
At the first hearing, and again last week, several Democrats on the panel questioned the council’s decision to propose national standards for student performance, but to leave standards for schools up to the states. (See Education Week, Feb. 12, 1992..)
The subcommittee is expected to hold a third hearing on the issue next month, before deciding whether to add legislation incorporating the council’s recommendations to a school-reform bill currently pending before the House. The Senate last month adopted legislation to create a new entity to oversee the development of standards and assessments.
Last week, much of the debate focused on the researchers’ concerns about the proposed system.
The O.T.A.'s report notes that, used properly, tests can be invaluable sources of information to help teachers and students in the conduct of learning; to monitor systemwide educational outcomes; and to inform decisions about the selection and credentialing of students.
Because of their utility--and relatively low cost--the use of tests has exploded in the past three decades.
Between 1960 and 1989, it states, revenues from the sale of tests used in elementary and secondary schools more than doubled in constant dollars, while enrollments increased by only 15 percent.
International Lessons
But with such increased uses have come abuses and misuses, the report states. Placing “high stakes"--such as student promotion and graduation--on test results has led to pernicious practices, the report notes.
Using the common analogy of tests as fever thermometers, the O.T.A. report contends that teachers, like doctors, often take actions designed to ease the symptom the high fever-rather than to cure the disease.
The value of testing, Mr. Feuer said, “has been compromised by policies that encourage excessive focus on improved scores rather than increased learning.”
Moreover, the report adds, using tests for purposes for which they were not designed--such as using college-admissions test scores to evaluate states’ educational programs--is rampant, and there are few safeguards against test misuse.
“Sadly, the historical misuse of tests has severely undermined their credibility even in applications for which they are well designed,” Mr. Feuer said.
National tests in other countries, meanwhile, offer some lessons for Americans, the o.T.A. report notes. However, such lessons “need to be considered judiciously,” it states.
A study of European and Asian exams-conducted for the agency by George F. Madaus, the director of the center for the study of testing, evaluation, and educational policy at Boston College, and Thomas Kellaghan, a professor of education at St. Patrick’s College in Dublin--found that such countries tend to test students at age 16 or later, and to use their tests for selection and placement, rather than for improving instruction.
Moreover, the exams vary widely in their use of multiple-choice formats and their allowance for local variations.
“To the extent to which we want to imitate our competitors,” Mr. Feuer said, “we have to choose which competitor we want to imitate.”
New Technologies
In an effort to improve the usefulness of tests, the report states, educators and scholars have developed new testing technologies, such as performance-based assessments and computer and video technology, aimed at tying testing to goals for learning.
But, it warns, “new assessment methods alone do not necessarily equip teachers with the skills necessary to change instruction and achieve new curricular goals.”
And, it says, such new technologies have not yet demonstrated that they meet technical standards for validity and reliability; that they can generalize student performance across a range of tasks; and that they can be fair.
Moreover, it points out, any change in designing, developing, and implementing tests would be expensive.
To gauge the current costs of testing, the O.T.A. obtained information from an unnamed urban school district. Direct outlays for standardized testing, the report states, totaled $1.6 million (in a $1.2-billion budget) in 1990-91, or about $6 per student. Adding indirect costs, such as teachers’ time to administer the tests, brought the cost to $1.8 million, or $13 per student.
Although reliable estimates of new assessments are unavailable, Mr. Feuer noted, the General Accounting Office has found that, in Ireland, it cost $135 to score a student’s responses to five essay questions in each of four subjects.
‘High-Powered Rifles’
Amplifying the O.T.A.'S concerns at the hearing here last week, Daniel M. Koretz, a senior social scientist at the RAND Corporation, warned that the standard’s council offered little evidence that its proposed system would avoid the problems of previous testing programs.
The recommendations, he asserted, “are unlikely to work, and may well have serious negative side effects.”
“if we’re doing something potentially risky and untried with nine million kids,” Mr. Koretz said, “we ought to find out what we’re doing.”
Governor Romer responded that he agreed that there should be additional research, but said the oversight mechanism should be put in place right away to keep the standards-setting process moving.
In addition, he added, the council intended in its report to correct some of the flaws of previous testing programs, not repeat them.
“I don’t understand why the high-powered rifles are aimed at what I think is a modest step aimed at getting something better,” he said.
Free copies of the summary of the O.T.A.'S report, “Testing in American Schools: Asking the Right Questions,” are available by calling (202) 224-8996. The full report is expected to be released next month.
