When experts want advice about how to devise or use tests, they typically turn to the 1985 Standards for Educational and Psychological Testing.
The standards have been revised four times since they were released in 1954, and are now undergoing a fifth revision. The latest document is likely to address many of the vexing technical issues raised by the current generation of student assessments.
“It would be my hope that people who were advocates for performance assessments would find in the standards that come out ways of judging the quality of their work and courses of action for them to take so that they would be able to improve what they’re doing,” said Eva L. Baker, a co-chairwoman of the 19-member committee that is revising the document.
Three national organizations drew up the standards: the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education.
Although the standards are voluntary, they hold great sway in the professional community.
The U.S. Education Department and its National Center for Education Statistics have adopted the standards, for example. And numerous federal laws, including the Goals 2000: Educate America Act, refer to them.
But, according to Robert L. Linn, a professor of education at the University of Colorado at Boulder, “serious questions have been raised about the degree to which the existing standards are adequate and appropriate for the wide variety of performance-based assessments that have been advocated in the past few years.”
Test Fairness
The committee is likely to address whether performance assessments--which ask test-takers to show their knowledge and skills through devices such as writing portfolios and hands-on experiments--should be held to the same rules for establishing validity and reliability as standardized tests.
“People are saying: ‘We want to harness the power of measurement-driven instruction for good rather than ill. We want to create tests that are good models of instructional activity. We want tests that teachers will teach to because what you assess is what you get,”’ said Edward H. Haertel, a professor of education at Stanford University.
“If the claim is put forward that this kind of testing will bring about salutary changes in classroom practice,” he said, “it becomes something that the test-validation process has to look at.”
One question is whether test- validation studies should examine both the positive and negative consequences of test use, primarily to insure that any negative impact on groups or individuals does not stem from a problem with the way the test is constructed.
The revised standards may also take a closer look at the use and interpretation of test results for specific populations.
No one knows, for instance, how the increased emphasis on written responses--even in mathematics assessments--will affect the performance of students whose native language is not English. And no one knows how to interpret the validity or accuracy of those results.
Similarly, the standards now provide little guidance about what kinds of accommodations are appropriate for students with disabilities or a limited command of English, when to use these arrangements, and how to interpret the scores.
Advice Being Sought
“The entire committee is aware of the importance of having these standards be user-friendly,” said Ms. Baker, “and address the kinds of issues that users come up against. And a lot of those have to do with differences in group performances and how one interprets that.”
“There’s a lot of controversy about what gets accommodated and where the line gets drawn,” she added.
Whether committee members will be able to agree on such topics remains to be seen. The committee first met in November 1993. Last October, 118 participants attended an invitational conference sponsored by the panel.
Assisting the panel are 225 advisers representing test publishers, advocacy groups, professional organizations, graduate programs in measurement and education, and the federal government. The advisers will be updated regularly on the committee’s progress and will review and comment on draft sections of the standards. The committee hopes to have drafts of sections ready for public review this year.
All three sponsoring organizations must approve the final draft before it is published, sometime late next year or in early 1997.
