Convening what the state schools chief called “the largest faculty meeting I have ever seen,” Vermont officials late last month brought together a 12th of the state’s teachers to brief them on their pioneering assessment program.
The mathematics and writing assessment, which is scheduled to begin on a pilot basis in about 135 schools next month, is being closely watched by educators across the country. It is the first statewide assessment to use portfolios, in addition to standardized tests, to measure student abilities.
Richard P. Mills, Vermont’s commissioner of education, said the project is aimed at providing parents and policymakers with a broader gauge of student knowledge and skills than traditional tests provide.
If it is successful, he said, it could encourage teachers and students to work together to raise the level of student achievement in the state.
“This is a vast conspiracy to boost the performance of every single child in Vermont,” Mr. Mills said. “What we’re looking at is not high test scores, but great schools.”
Many of the 500 teachers who met here appeared enthusiastic about the project. Officials noted that, although the state had chosen only 48 schools to participate in the pilot project, nearly 90 additional schools asked to join.
Several teachers expressed concern, however, that current school schedules offer too little time for them to work with students to develop portfolios.
And some said they feared that they would be evaluated on the basis of the student work.
“This is a powerful tool to help children,” said Franklin J. Chrisco, a 5th- and 6th-grade teacher at Brattleboro Elementary School. “I don’t want teachers to avoid its use.”
But Grant Wiggins, an assessment consultant based in Rochester, N.Y., told the teachers that the assessment system could prove to be a “Trojan horse” that could force changes in school structure and governance.
“This is the first systematic attempt to try out the real arguments of school restructuring--to adapt schools to meet the needs of learning,” he said.
‘Dragged Me Into This’
First proposed by Mr. Mills and adopted by the state board of education in 1988, the assessment program would be the first statewide test in Vermont’s history. (See Education Week, Oct. 26, 1988.)
Although the plan had the strong support of Gov. Madeleine M. Kunin, it initially drew a skeptical response in the legislature, according to State Representative Barbara Grimes, chairman of the House Education Committee. Legislators expressed fears that the test would usurp local control, she said, as well as concerns about the novelty of portfolio assessment.
As a result, she said, the legislature pushed back implementation of the test by a year, while allowing state officials to continue its development.
“They literally dragged me into this,” Representative Grimes said. “How can a group of teachers give the same kind of grade on a portfolio?”
But business leaders impressed upon lawmakers the need for information about student performance, she said, and teachers convinced them that portfolios would be preferable to conventional tests. This year, the legislature agreed to provide $425,000 for the program, even though lawmakers cut funding in other areas of the budget.
“The alternative is a Regents-type examination,” the committee chairman said, referring to New York State’s standardized tests for high-school students. “The teachers made it clear they don’t want that.”
The assessment may be expanded in the future to include other subjects, such as science and the arts, and to include students in the 11th grade, Ms. Grimes noted.
“My only regret,” said Governor Kunin, who is retiring at the end of this year, “is that I won’t be around to brag about the results.”
Mixed Strategy
Under the plan developed by teams of teachers and state officials over the past year, the state will assess students in the 4th and 8th grades in writing and mathematics in three ways:
A uniform test, which uses equivalent tasks administered under the same conditions for each student. Although the test has yet to be developed, it is expected to be largely performance-based, rather than multiple-choice.
A portfolio, which includes material collected during the course of the year.
A “best piece,” which represents what a student considers his or her best effort for the year.
The mix of assessment instruments will provide a “safety net” to ensure that the program does not become obsolete, according to Mr. Mills.
“Assessment is changing so fast in this country,” he said. “We felt we needed a mixed strategy.”
The different pieces will also enable the public and policymakers to hold schools accountable for student performance, the commissioner noted.
In addition to serving as a standardized instrument that would permit comparisons of performance across schools, he said, the uniform test is expected to include items from the National Assessment of Educational Progress, which would enable state officials to compare Vermont students’ performance with that of their peers nationwide.
“That’s very important to the state board and the legislature,” he said.
Results from the portfolio and “best piece” assessments, Mr. Mills said, will be reported in a “school report day,” a kind of town meeting that will permit parents to see their children’s work and its evaluations. Although during the pilot year the state will only provide statewide results, the school report day is expected to become an annual event for each school, he said.
“Now, conversations about student performance are ethereal,” he said. “What does a 550 on the Scholastic Aptitude Test mean?”
By contrast, Mr. Mills said, the report day “is intended to present information to the public about a school’s goals and results.”
“People can not only hear about the results,” he added, “they can see and touch the work students are doing.”
Two Portfolios
The heart of the system, and what has attracted national attention, is the portfolio.
Teachers participating in the pilot will next month begin collecting samples of student work that will include finished papers and drafts, which are expected to show how students approached problems and tried solutions.
The writing portfolio is expected to contain a poem, play, or personal narration; a “personal response” to a cultural or sports event, book, mathematics problem, or current issue; and prose pieces from classes other than English and language arts.
The materials will be evaluated on at least seven criteria, including the degree to which the organization suits the writer’s purposes, the writing exhibits a sense of personal expression, and the use of detail adds to clarity, as well as evidence of progress over time and evidence of opportunities for students to revise their work.
The math portfolio is expected to contain a range of materials that ''demonstrate the student’s ability to learn and understand materials beyond the ‘facts and knowledge’ level,” according to a report by the math committee.
Such materials, the report states, could include: a solution to a problem assigned as homework; a problem made up by the student “with or without a solution"; a paper done for another subject that includes math, such as an analysis of data presented in a graph; or entries from a journal.
Kathy Ernst, a teacher from Brattleboro Elementary School, said the math committee has not yet reached a consensus on the criteria for evaluating the portfolios.
But she added that the portfolio may include nonwritten materials, such as videotapes or audiotapes, so that evaluators can focus on students’ mathematical skills.
“You can’t assume that just because kids write well, they are the best mathematical thinkers,” she said.
Higher Standards
By including a range of materials and a range of criteria for evaluating them, portfolios would provide more complete information about student abilities than conventional tests, said W. Ross Brewer, director of planning and policy development for the Vermont Department of Education.
“This is a much more complex way of looking at performance,” he said. “This shows what are the real things people can do, and where they are coming up short.”
At the same time, Mr. Wiggins noted, the process of putting together the portfolios would encourage students to strive to a high standard. By contrast, he said, traditional tests encourage a “compliance mentality.”
“We don’t produce a lousy car and wait for the consumer to judge it,” Mr. Wiggins said, “yet we do that all the time with students.”
“Many people see what you are about to do,” Mr. Wiggins told the teachers here, “as the opening shot in a campaign to see if we can have standards without standardization.”
The “best piece,” Mr. Mills added, would also encourage students to produce high-quality work by offering them an opportunity to assess themselves.
“That’s a powerful question--'what is my best piece?”’ Mr. Mills said. “In answering it, students have to reveal their own standards.”
‘Cart Before the Horse’?
Teachers here generally praised the program and said they supported its aims.
“It’s good not just to chronicle students’ progress as writers and mathematicians,” said Jim DeFilippi, an English teacher at Winooski High School. “It really feels good to have physical proof that that happened.”
But others warned that the kind of student-teacher interaction that envisions is difficult under current school structures, where teachers see as many as 120 pupils a day.
Several participants said, too, that they worry they may be held accountable for student performance, even though they have scant opportunities to influence that performance.
As one teacher put it, the assessment may be “putting the cart before the horse.”
But Mr. Mills responded that the new program will provide a more accurate picture of performance than schools currently provide.
“The public already holds us accountable, but to the wrong things,” he said. “They have a picture of what goes on that’s totally wrong.’'
He added that a key feature of the program is staff development, and noted that the department will be holding a series of regional meetings, as well as providing a roster of teachers who can provide inservice help to teachers. The University of Vermont has also established a toll-free computer bulletin board to enable teachers to ask questions and share information, he said.
Mr. Mills added that the assessment this year is a pilot program that could be adjusted if it proves unworkable. But, he said, he is confident that it will succeed in raising the level of student achievement.
“What we are doing is very risky,” the commissioner said. “Nobody else is doing this.”
“But it is the right thing, and it will work,” he said.
