In Pittsburgh: Tracking Arts 'Footprints' Toward New Type of Test
Pittsburgh--The 7th-grade violinists listen to two recorded versions of "Chop Sticks"--one their first sight-reading of the piece and the other a rehearsed performance several days later.
Then they bend over the sheet of paper in front of them and begin to write: What was good or bad about the performance? What got better the second time? How could the ensemble improve?
The exercise is part of an experimental research program in music, the visual arts, and creative writing undertaken by Har6vard University's Project Zero, the Educational Testing Service, and the Pittsburgh Public Schools.
Known as Arts propel, the project's goal is to create curricula and assessments that provide a much richer depiction than is now available of how children learn and grow artistically.
Today, most student tests are given at the end of a semester or school year. And they provide little guidance for teachers on how to change their day-to-day activities.
In contrast, propel is designing assessments that are woven into the warp and woof of daily classroom instruction.
As students in propel produce sketchbooks and journals, compile portfolios, and complete carefully sequenced classroom exercises, they leave behind a series of "footprints" for teachers about how they are growing and thinking as artists.
The hope is that the more rapid, qualitative feedback provided by these assessments will prove more meaningful to students and teachers than current tests.
In addition, because propel exercises double as instructional tools, they are helping to modify the curriculum in classrooms and improve the quality of student discussions.
In the last few years, experts have criticized standardized tests for failing to capture the more complex aspects of learning. These include the ability of students to pose interesting questions, devise novel solutions to problems, and create original works.
Others contend that the excessive weight placed on standardized-test scores has served to routinize instruction and impoverish learning.
"If you want to change what goes on in schools," argues Howard Gardner, co-director of Arts propel and author of Frames of Mind: The Theory of Multiple Intelligences, "you must change the way things are assessed."
"As long as we use standardized, multiple-choice, machine-scored tests," says Mr. Gardner, who also co-directs Project Zero, "that's going to dominate what goes on in the class."
In a handful of places, educators who have taken Mr. Gardner's argument to heart are trying to develop broader, more qualitative measures of student learning.
Last month, for example, Vermont's superintendent of education proposed using a combination of test scores and folders of students' work to measure achievement in writing and mathematics.
Of the recent initiatives, however, Arts propel represents the most un4likely combination of partners. It has brought together the largest purveyor of student tests, practicing classroom teachers, and one of the country's most avant-garde thinkers about intelligence.
The project, which is supported by more than $1 million from the Rockefeller Foundation, is still in its formative stages.
But in the next three years, Pittsburgh officials hope to replicate it in all of their middle and high schools.
"I believe that what gets tested is what gets taught," says Richard C. Wallace Jr., superintendent of Pittsburgh's schools. "If we assess or gather different kinds of data, then we're going to force different kinds of interactions in the classroom."
According to Mr. Gardner, schools now overemphasize a narrow spectrum of human thought focused on linguistic and logical abilities.
They pay little or no attention to other mental capacities, such as musical intelligence, spatial intelligence, or interpersonal skills.
By focusing propel on the arts, the Harvard researcher and his colleagues hope to bring these varied human intelligences to the forefront and learn more about the artistic development of children. In doing so, they may also elevate the place of arts education in general.
In addition, Mr. Gardner notes, propel has the potential to provide ''a place in the sun" for students who do not traditionally excel in schools, but who may be "more imaginative or divergent thinkers."
The ets and Harvard researchers chose Pittsburgh as the site of their research because of the school system's strong commitment to testing and to arts education. In addition, the district has a cadre of skilled arts teachers who have worked with the researchers side-by-side in designing the project.
Propel emphasizes a particularly broad definition of artistic learning,8designed to be accessible to all students.
In propel, they are asked to create original artwork, look carefully at what they and others have produced, and then step back and reflect on that work over time.
Those three aspects of artistic thought--production, perception, and reflection--are present in all of the project's major activities: student portfolios, student sketchbooks or journals, and "domaints."
A domain project is a sequence of related exercises that introduces students to a central concept, question, or problem within an artistic discipline.
At the same time, each domain project is carefully designed to generate samples of students' work that inform teachers about their progress.
In a series of exercises on poetry, for instance, Diane Hughes, an English teacher at Carrick High School, begins by having her students read some poems that are built around lists of nouns or related ideas.
The students talk about what differentiates these poems from other forms of literature; speculate on why the poet chose to order the list as he or she did; and explore the meaning of basic concepts, like rhythm, pattern, and repetition.
On subsequent days, students may be asked to write similar poems in groups of four, or to insert their own lines into an existing poem.el20lLater, they are asked to write a "list" poem that reveals the power, mystery, or magic in an object.
Throughout the project, the students stop and discuss what they are doing, write about it in their journals, read their poems to each other, and critique and evaluate what they have heard. Successive drafts of their poems are also collected to trace their development over time.
Other domain projects are focusing on composition and style in the visual arts; monologues in creative writing; and notation, individual performances, and group rehearsals in music.
Within each domain project, teachers and researchers are trying to identify key dimensions of student learning that would help tell them whether students have understood a concept deeply and thoroughly.
In the domain project on "music reading," for example, teachers rate students on such "performance" dimensions as their ability to exercise finger control, to use rhythm accurately, and to grasp the expressivity and style of musical phrasing.
But they also assess how carefully students can listen to and think about their music--whether they can critically compare two separate performances of a piece and suggest strategies for improvement.
The formal scales for assessing such dimensions are still being developed. Eventually, achieving some common, reliable means of rating students on each dimension will be necessary, Mr. Gardner notes, if methods like the domain projects and portfolios are to be used at the national, state, or district level.
But propel's primary interest, according to Drew Gitomer, director of the project at ets, is to provide qualitative information about an individual student's strengths and weaknesses that would help teachers in the classroom.
Moreover, by highlighting those features of a discipline that teachers themselves have identified as most important, he says, the exercises are serving to focus and improve classroom teaching.
The domain projects are essentially a "condensation" of what good teachers already do, according to Dennis Palmer Wolf, a research associate with Arts propel.
"A good teacher knows that one assessment doesn't yield a kid any sense of his or her own progress," says the Harvard researcher. "One thing that you should always be doing is creating behind a student a set of footprints about his growth."
As the domain projects build on exercises familiar to the schoolroom, the "portfolios" employ a device common to the art world.
But unlike the slick representa4tions of finished pieces that make up most artists' portfolios, the folders being developed in Arts propel contain sketches, works-in-progress, self-evaluations, classroom exercises, and journal entries.
The researchers hope that the detailed record created by such folders will enable teachers to look at longitudinal changes in a student's thinking and development that could not be captured through more limited classroom exercises.
Moreover, because students have some control over what goes into their portfolios, the expectation is that they will become more thoughtful about and involved in their work.
Pittsburgh teachers traditionally have kept folders of students' work in both writing and the visual arts. But the teachers involved in propel say it is changing how they use that information.
"Before, the portfolio was probably used for management's sake," says Karen R. Price, an instructional teacher-leader in the visual arts at Schenley High School. "We stored everything in the portfolio so it wouldn't be all over the room."
"Now," she says, "we're talking with kids about their portfolios as they're doing things. We're pulling things out of their folders and saying, 'Look, this is what you did here. What can you do from this point? Where do you see an area of need?"'
To date, the project has not tried to formally score or assess the work in a student's folder, Mr. Gitomer said. But the rich materials gathered through the portfolios should provide teachers with the basis for more informed decisionmaking.
Like the portfolios, student sketchbooks and journals are intended to make visible the more elusive aspects of learning, such as how students feel about their work and what interests them.
The sketchbooks produced by Norman Brown's visual-arts students at Schenley High School include pen-and-ink sketches, cartoon drawings, notes from friends, reproductions of museum pieces, favorite photographs and advertisements, and running written dialogues between Mr. Brown and his students about their work.
In addition, propel is exploring the use of "biographies" that would trace how a student's ideas about a single piece of artwork change and develop over time.
"Kids have a lot to say," notes Mr. Brown, "and I'm now allowing that to come out through their artwork, rather than me doing all of the thinking for them."
The result of that extended dialogue, he suggests, is the beginning of a "studio environment" in his classroom, "where it was always in my mind just a class, a 45-minute period."
Other teachers say that the focus on higher-order thinking in propel is altering and improving classroom instruction.
"I'm finding that I'm having much more enriched conversations about a topic that I love," says Barbara Albig, a visual-arts specialist at the South Vocational Technical School.
"Rather than me asking the questions, students are asking questions of themselves," she says. "You essentially are giving them ownership and responsibility for the whole process. And that's a big shift."
Jim Charlton, a choral-music teacher at Langley High School, says he is purposefully trying to create more opportunities for reflection and discussion in his classroom, "even if it slows down the rehearsal."
"Propel encourages a lot of higher-level mental activities that simply imitating the teacher--getting up and performing--doesn't provoke,'' he explains. "Many of the students that public schools turn out are really sort of clones. They do what they think the teacher wants, and they don't ever really take the time to reflect for themselves on what's good or bad."
Arts propel, he argues, "is going to encourage the students to become self-directed not just in the arts, but in anything they do."
At present, however, propel is still in its infancy, with both teachers and researchers struggling to make their ideas a reality.
The creative-writing teachers, for instance, are just beginning to work on the notion of portfolios this year. And in the visual arts, Ms. Albig says that "domain projects" are still the "weakest link of our whole project."
"They will eventually be one of the strongest links," she says. "In the long run, these will be developmental activities that pose a problem, have multiple solutions, and cause students to do some critical thinking."
But for now, Ms. Albig notes, many of the domain exercises are too broad and too confusing for stu4dents, so that "some of what we're intending to build can get lost." A domain project on "style," which the art teachers worked on last year, will be revised again this year with such criticisms in mind.
Moreover, those involved in the project are still trying to overcome students' reluctance to save works-in-progress, to write consistently in journals, or to move beyond right-and-wrong answers.
Several teachers, for instance, talk about regularly fishing students' efforts out of wastebaskets and restoring them to portfolios.
Notes Mr. Gardner: "There's nothing that we're doing now that we're sure we're going to continue doing."
Researchers also are just beginning to tackle the sticky problem of how to use the rich collection of student data that has been generated to arrive at more formal judgments.
Last month, a group of Pittsburgh teachers, practicing artists, and art educators met in Princeton, at the home of the Educational Testing Service, to begin looking for a reliable way to assess the work coming out of the domain projects.
"What we are looking for is some sort of profile of a student," says Mr. Gitomer of ets, "so that if you say a student is a '4' on a particular dimension, you have a reasonably decent understanding of what that means."
But he cautions that these alternative forms of assessment cannot be analyzed using "traditional psychometric methods." Testing experts, he suggests, must be prepared to make some "compromises."
In fact, the strength of Arts propel may be its ability to move beyond a purely psychometric model of testing to one that is more useful for practitioners, says Paul G. LeMahieu, director of testing and evaluation for the Pittsburgh schools.
"The psychometric model is so strong and does such a good job of disciplining the data that it's almost as if it's been providing us with the very best, highest-quality data available," Mr. LeMahieu says, "only it turns out to be the wrong stuff."
"What we need," he argues, "is a way to systematically and routinely squeeze assessment information out of instructional events ... so that it will be a whole lot more informative about clinical practice."
Arts propel, he and others assert, is providing a promising means of doing that.