Supply of New Assessment Methods Said Trailing Behind Strong
Demand By Robert Rothman
Manhattan Beach, Calif--The strong demands by policymakers at the national and state levels for new forms of student assessment are running far ahead of the available supply, researchers said at a conference here this month.
Speaking at a meeting organized by the federal Center for Research on Evaluation, Standards, and Student Testing, participants noted that at least half a dozen national groups have called for some form of new national test or examination system, and that others have proposed substantial changes in the tests that currently exist.
At the same time, several states have also mandated a rapid shift from conventional tests to performance-based measures of student learning, speakers noted.
But researchers here, while generally supportive of performance-based assessments, were skeptical that they could be implemented as policymakers are demanding.
"Just because you want to have it doesn't mean you can have it," said Eva L. Baker, co-director of the research center, which is based at the University of California at Los Angeles.
Ms. Baker said it will take 5 to 10 years to demonstrate that the new assessments measure what they are intended to gauge.
During that time, she added, teachers must also be trained to transform instruction to match the assessments' goals.
"There is skepticism over whether it's going to work," Ms. Baker said. "How could teachers possibly find the time to learn all this new stuff on the fly?"
But Jack Foster, education aide to Gov. Wallace G. Wilkinson of Kentucky, said there is a sufficient body of knowledge to begin implementing the new measures, even if they have to be modified over time.
"Performance assessments were not invented in 1988," Mr. Foster said. The policymakers "are catching a lot of statisticians unawares."
"Even if it's ultimately not what we want," he added, "[what could be in place this year] is a lot better than what we have."
The conference here was called to inaugurate the center on assessment, the largest of the 17 federally financed research centers awarded last year by the U.S. Education Department.
Christopher T. Cross, assistant secretary of education for educational research and improvement, said the center's agenda has become more critical in recent months, as a broad range of groups--including President Bush's advisory panel on education policy, the National Education Goals Panel, and the National Center on Education and the Economy--have recommended that new measures of student performance be created.
"How can we advance toward national, state, or local goals without an adequate picture of the most important feature in any school system? We cannot," Mr. Cross said. "That's why several groups have recently called for new and better tests of students."
But Marshall S. Smith, dean of the graduate school of education at Stanford University, said there is confusion about what the proposed assessments are expected to accomplish.
They have been promoted, he noted, as everything from a "wake-up call" that will alert parents about their children's achievement to "authentic representations of student work."
With such different, and at times conflicting, goals, Mr. Smith said, "there is a great risk they will not accomplish any purposes at all."
Moreover, added Sylvia T. Johnson, professor of psychoeducational studies at Howard University, the test proposals, by themselves, are unlikely to affect student performance.
"Just changing the assessment," she said, "is not going to fundamentally change what happens in schools unless we make deliberate steps in that direction."
In addition to questioning the assessments' effect on schools, speakers here also expressed concern over implementing them.
Several participants noted that there is little evidence of public support for creating new measures of student performance.
Referring to the motion picture, Dena G. Stoner, executive director of the Council for Educational Development and Research, said some advocates of assessment reform express the "'Field of Dreams' concern--if you build it, they will come."
Emerson J. Elliott, acting commissioner of the National Center for Education Statistics, also pointed out that considerable research is necessary to demonstrate that performance assessments are valid and reliable measures of student achievement.
"Performance assessment is enormously appealing to the policy community," he said. "But there is a gap between the concept and the ability to put it in place for national statistical purposes. There are questions of how to create them, how to judge the results, and how to judge them consistently."
The research must also show that the new measures are more fair than existing tests to members of minority groups, who tend to perform less well than white students, noted Linda F. Winfield, principal research scientist at the Center for Research on Effective Schooling for Disadvantaged Students at Johns Hopkins University.
"You can't make the assumption alternative forms of assessment will prevent unfairness," she said.
Ms. Baker and other assessment-center researchers pointed out that the center plans to develop prototypes of new assessment techniques, as well as guidelines to permit states and districts to evaluate them. The center is also expected to work directly with states and districts as they develop new measures of student performance.
If it is successful, she added, the research would show that the alternatives can improve assessment and, ultimately, schooling.
The center's previous work, which produced a model performance assessment in history, shows that such models can be effective, Ms. Baker said.
"We did it," she said. "The things work."