When the state that pioneered portfolio assessments decided last fall to revamp its testing system, some educators and policymakers asked whether Vermont was backing away from what looked to be a promising, albeit challenging, method of gauging student performance.
But Vermont’s plan to mix more traditional measures, including multiple-choice questions, with its avant-garde approach to testing reflects an increasingly popular view.
Delaware, Kentucky, and Massachusetts are among the states that are also changing their existing assessment systems or constructing new ones. They are doing so with an eye toward producing a coherent compilation of different ways to measure student progress--an approach known as “multiple measures"--in an effort to get a better picture of how much and how well students have learned their lessons.
Educators have only a few pieces of evidence, collected through assessment, on which to base inferences about student learning, said Ed Reidy, the director of assessment for Kentucky. “If you have different types of evidence,” Mr. Reidy said, “you have a stronger inference.”
The idea that states should use multiple measures in concert marks a departure from the belief that test-makers must take sides and embrace either machine-scorable multiple-choice exams or open-response or performance tests. (“State Test Questions Focus of Renewed Scrutiny,” Jan. 31, 1996.)
It’s “a whole new frontier,” said Dennie Palmer Wolf, a senior research associate at Harvard University’s graduate school of education. What states like Kentucky and Vermont are doing, Ms. Wolf said, represents “what we always wanted, which was a diverse set of measures. We now have people working hard at figuring out what each of these assessment instruments does best.”
Advocates of a multiple-measures approach include New Standards, a Washington-based coalition of states and urban districts led by the National Center on Education and the Economy and the University of Pittsburgh, that has been working since 1991 to create high performance standards and a parallel assessment system. Delaware, Kentucky, and Vermont are all members of the consortium, as was Massachusetts until last year. New Standards developed the standardized assessments that Vermont is planning to add to its portfolio system.
“New Standards’ contribution was definitely to open up the education community and the assessment community to the uses of performance assessments,” said Wayne H. Martin, the director of the state education assessment center at the Council of Chief State School Officers in Washington. And New Standards’ advocacy of a balanced or mixed-measures model, he said, “is good because it’s confirmation for the direction many states are going.”
The concept is also important because of the federal Title I program, which provides money for remedial education for disadvantaged students. Revamped by Congress in 1994, Title I requires annual evaluation of student progress and the use of multiple measures. (“State Progress On Title I Rules Called Uneven,” April 10, 1996.)
No Backing Away
Vermont made the leap to a voluntary portfolio-assessment system in 1988. Some 90 percent of the 371 schools in the state currently use portfolios, although they have not been required for state testing. Kentucky was one of the few states to join Vermont in using portfolios, or collections of student work over time, for statewide assessment. Others, however, have been closely watching the experiment.
But when this leading proponent of portfolio assessment changed its policy in November, the questions from inside and outside Vermont were so plentiful that the state education department issued a memo stating that neither the department nor the state school board was “backing away from portfolios.”
“The board feels [the use of portfolios] is a cornerstone of assessment, that it provides rich, deep information about student work,” Doug Walker, the director of Vermont’s school and instructional support team, said.
From the time it instituted portfolios, the state planned to add other types of assessments and to make a whole system that is “effective, efficient, and affordable,” Mr. Walker said.
“Our interest at the statewide level,” he said, “was to build a comprehensive assessment system that provided a variety of groups the type of information they need about student performance.”
Even though they serve as portfolio evaluators, Vermont’s teachers have also worried about the exclusive use of portfolios to gauge student learning, especially when it has been hard to ensure that one portfolio rater has the same view of academic achievement as another.
“They’ve felt unfairly victimized by comparative rankings [when] they know themselves sometimes it has been challenging to achieve ... reliability across schools and school districts, let alone across the state of Vermont,” said Angelo J. Dorta, the president of Vermont-NEA, the state’s teachers’ union.
Multiple-Choice Reappears
Kentucky officials also have been adding assessment elements, shelving others, and trying to ease the burden of statewide testing for students and educators. Unlike Vermont’s system, Kentucky’s assessments are so-called high-stakes ones. They are used to determine cash rewards for schools or penalties such as state supervision.
Kentucky has also returned multiple-choice questions to the lineup, and in so doing has reduced the number of essay questions. Multiple-choice items had been used before in the state assessment, though not recently and not for accountability purposes.
Mr. Reidy said that multiple-choice questions were put back primarily because they allow the test givers to cover more content more quickly. They are also less expensive to devise and score than are open-response items.
In a particularly controversial move, Kentucky officials have dropped the popular “performance events” from the state assessment to examine whether they are equivalent from year to year. The hands-on tasks require small groups of students to conduct science experiments or figure out, for example, where to locate a fictional industrial plant.
Kentucky had also been evaluating student portfolios in writing and mathematics. But last year, the legislature decided to pull temporarily the math portfolio for further study because the student workload was too burdensome. The writing portfolio remains alive and well.
The burden to students and teachers has also been reduced by spreading testing over six grades instead of three, cutting in half the classroom time devoted to testing in any one grade, Mr. Reidy said.
While it will not bring sanctions or rewards, this spring Kentucky 3rd, 6th, and 9th graders will take the norm-referenced Comprehensive Test of Basic Skills in reading and math. The state added the CTBS, Mr. Reidy said, to satisfy parents and others who want to know where their children stand against students nationwide.
Graduation Requirement
Two other states also are creating new statewide assessments.
Massachusetts’ system, to take effect next year, is expected to include both multiple-choice and open-response portions. And it will be high-stakes: Beginning with the class of 2003, students will have to pass the 10th grade test before they can graduate.
An assessment system using multiple-choice items alone would not have the same positive impact on classroom instruction and “can’t be the basis of high-stakes state tests of the future because of a likely successful legal challenge by a student who failed,” said Alan Safran, a spokesman for the state’s education commissioner.
When Delaware gets its new assessment system under way later this year, the state plans to use a variety of open-response questions and hands-on performance tasks, said Rebecca Kopriva, the state’s director of assessment.
The state also wants statewide assessment that is accessible to students with diverse ways of learning. That means, for instance, asking questions about videotaped clips and using plain language, Ms. Kopriva said.
“There are a lot of ways you could simplify the literacy portion of science exams,” she said. “Often you have compound sentences, mixed up tenses. ... There’s no need to do all those things. You can make it very straightforward.”
