Copyright 1985, Editorial New York--The Educational Testing Service will devote $30 million over the next 15 years to the search for a “new generation” of technologically sophisticated tests that instruct as well as assess, ets officials announced here late last month.
The initiative, called “Project Jessica,” will create testing and measurement instruments that are primarily computer-based and are more integrated with classroom instruction than current standardized tests, said Gregory R. Anrig, president of the organization.
The project is named for a 4-year-old girl whom Mr. Anrig declined to identify, saying only that she represents a generation of students who will live out most of their lives in the next century.
Announcement of the project came at the annual meeting of the ets, where the program theme--"the redesign of testing for the 21st century’'--was further underscored by predictions that advances in technology and the cognitive sciences would revolutionize the field within the next two decades.
Current tests are primarily “one-shot” assessments, said Mr. Anrig. They compare the performances of test takers on the basis of a single evaluation, and they provide information too late for it to actually aid in changing behavior.
The future assessments to be developed by the ets, he said, will provide more information on how students are doing, where they are making mistakes, and how teachers should go about correcting student deficiencies. Such measurements will not, however, eliminate the need for admissions examinations and other sorting instruments, Mr. Anrig said.
‘More Useful’ Tests
Robert Glaser, professor of education and psychology at the University of Pittsburgh and director of its learning-research center, told conference participants that by the year 2000, tests may be able to analyze why students make the errors they do.
Cognitive research, he noted, has found that student errors often are not the result of random or careless behavior, but stem from basic misunderstandings about the task at hand.
He cited the example of a student who consistently uses the phrase “in which” to connect a relative clause at the end of a sentence with the one preceding it. The identification of such systematic errors, Mr. Glaser said, may help teachers see students’ logical--if incorrect--thought processes and then redirect key aspects of their instruction to correct them.
Such assessments--which would provide direct guidance to classroom teachers--represent the most hopeful direction for future test development, said George F. Madaus, director of the Boston College center for the study of testing, evaluation, and educational policy.
Computer-developed programs that check spelling and grammar, count types of words, and monitor syntactical usage can help teachers improve students’ vocabularies and writing skills, he said. And the use of computer graphics, video documents, and “digitized” photography, music, and speech will make it possible to present students with a “wide range of stimuli” and assess their progress in a more open-ended fashion.
Mr. Madaus also suggested that banks of test items, kept on a central computer, would enable teachers to quickly design tests to meet particular needs or to give students practice in certain skills areas.
‘Adaptive’ Testing
One of the new forms of testing--computerized “adaptive” tests--is already a reality, said William C. Ward, director of measurement-technology research for the ets
An adaptive test is one in which questions are weighted according to the level of skill required and students proceed on the basis of their demonstrated proficiency.
In a full-fledged adaptive test, Mr. Ward explained, the test taker is evaluated after each answer. The computer selects and administers the most appropriate next question based on this evaluation.
Adaptive tests may require only half as many questions as conventional examinations, he said, yet yield the same or better information.
During the past 18 months, the ets and the College Board have been trying out a pilot adaptive test of basic skills as a possible tool in making college-placement decisions.
Rapid advances in computer technology should help speed the development of adaptive-testing programs, Mr. Ward noted. When the ets began its research in the field, he said, “the computer needed to deliver such a test sold for more than $3,000.” Now, he said, “we can now do quite nicely with an off-the-shelf personal computer that retails for about $600.”
He predicted that adaptive testing “will become the norm in large programs of standardized testing, particularly those that emphasize ‘academic’ aptitude and achievement measurement.”
Barriers to Reform
But beyond such technical innovations may lie serious conceptual barriers to true test reform, according to Robert L. Linn, professor of education at the University of Illinois at Champaign-Urbana. In his view, the search for more productive forms of testing must involve not only methods but also deeply ingrained attitudes about how tests are used.
Boston College’s Mr. Madaus also expressed concern over this issue, noting that a growing number of state testing programs are designed merely to sort and select both students and teachers and do not incorporate any of the new avenues of assessment available.
“In testing,” he maintained, “the temptation to relegate the human factor to minor significance is a growing trend.”
“The trend is most visible,” said Mr. Madaus, “in the administrative use of multiple-choice tests to make automatic policy decisions about such things as high-school graduation, grade-to-grade promotion, teacher certification, and merit pay.”
“The preeminent peril latent in the new technologies,” he concluded, “is that they can further strengthen the illusion that test results are totally significant, absolutely rational, objective, definitive, and right.”
