Pilot Test Offers a Glimpse of Board's Teacher Assessment
TAMPA, FLA.--Since the National Board for Professional Teaching Standards was founded five years ago, educators have wondered what its assessments for identifying superior teachers would look like.
Now, as the developers of the board's first assessments move into the final stages of their work, a picture is emerging of the types of exercises teachers may be asked to do.
The University of Pittsburgh and the Connecticut Department of Education, which are creating the assessments for teachers of English language arts who work with children in early adolescence, are proposing a three-part package.
One component is a portfolio, a tool that increasingly is seen as holding great promise for assessing the work of teachers and students alike.
Over the summer, a group of educators gathered at the University of Southern Florida here to score sample portfolios that were assembled by classroom teachers.
For many participants, the intensive, three-week session provided the first opportunity to learn the details of the national board's vision of accomplished teaching. It was also a time to take stock of their own teaching in light of the practices valued by the board.
The experience was eye-opening for Raymond W. Fones, a language-arts teacher in Zephyr Hills, Fla.
"There are things in my classroom I'll never do the same again,'' he said.
The Pittsburgh-Connecticut team is proposing that candidates for certification undergo a written test of their "content knowledge''; assemble a three-part portfolio from their real-life classroom work and experiences; and participate in a series of exercises at an assessment center.
After the assessment developers submit their final report next year, the national board will make the final decision about which exercises to use.
Along with the portfolio exercises, the written examination also has been field-tested. The assessment-center component will be tested this fall.
Costs and Incentives
The 30 judges who gathered here were assembled with the help of the Florida Council of Teachers of English and were paid $100 a day for their time.
Despite their willingness to undertake the work, some of the judges said they were worried about the cost of undergoing the assessments and the incentives for doing so--questions that have been raised for years about the system of voluntary national teacher certification that is now taking shape.
But the judges also were eager to be part of an endeavor that they believe will bring their profession much-needed respect.
A system of national certification "is a necessity,'' said Carole Scala, a teacher in Orlando, Fla. "This will give our profession the credibility it's lacking. The public needs concrete proof that we're a profession.''
National certification alone is unlikely to do that, she added, unless extra money is provided for teachers who meet the criteria.
The training and scoring session was led by Penny Pence, a 7th- and 8th-grade English teacher from Kittanning, Pa., who has worked full time at the University of Pittsburgh on the development of the assessment, and several colleagues.
Ms. Pence began by telling the judges that she recognized "there are a lot of issues associated with the idea of testing our peers, and some rather large implications for our profession.''
"I made the commitment because someone is going to do it,'' she explained. "I have come to believe in the profession even more because of this project. It has put me in touch with so many fine teachers.''
The judges spent the first week here learning the proposed scoring system for the portfolio exercises.
The next two weeks were devoted to evaluating the 75 portfolios completed by teachers across the country. The same teachers also completed the written portion of the assessment.
Of the 300 teachers who agreed to take part in the exercise, only about 75 finished their work, Ms. Pence said. She attributed the drop-off mainly to the time constraints the teachers worked under, noting that they had only four months to assemble the portfolios.
The laboratory plans to recommend to the national board that actual candidates for certification be given a year or two to create their portfolios, Ms. Pence added.
The portfolio segment of the proposed assessment has three
"Planning and teaching'' tracks a teacher's work over three weeks, using journal-style commentaries written by the teacher, videotape, student work, classroom handouts, and other materials to examine how the teacher plans and thinks about her work.
In another exercise, "student learning,'' the teacher is asked to prepare folders showing two students' writing and to write about her goals for the students and how her instruction influenced their development.
The final segment is a videotaped "interpretive discussion'' in which the teacher and her class talk about a piece of literature they have all read.
Scoring Tied to Standards
The three portfolio exercises, the written examination, and the assessment-center work are all designed to examine five critical aspects of a teacher's work, Ms. Pence said.
These scoring "dimensions'' are themselves based on the standards for accomplished English-language-arts teaching that are being derived by a committee appointed by the national board.
The assessment package looks for evidence that a teacher is: "learner centered,'' basing instruction on her knowledge of the students; aware of factors like culture, gender, race, and socioeconomic status that influence students' learning, and able to teach accordingly; knowledgeable about her subject matter; able to integrate the components of language-arts instruction and also mesh it with other subjects; and skilled at using "coherent pedagogy'' to achieve her aims.
Not every exercise is designed to reflect all five dimensions. The judges who scored the discussion exercise, for example, were trained to look for evidence in three categories: learner-centeredness, content knowledge, and coherent pedagogy.
To score the teachers' portfolios, the judges were trained to gather evidence before drawing conclusions. To do this, they were given a set of guiding questions for each exercise.
The dimensions are not "perfect boxes,'' Ms. Pence observed, "but more like filters.''
Most of the time, the judges worked in pairs. They were assisted by six "lead judges'' who had been previously trained and who helped answer questions.
The judges were asked to review the evidence, take notes and organize their data using the guiding questions, and then meet to discuss their findings and reach agreement on the evidence they found for each dimension.
They were then asked to write a summary of the evidence for each dimension and assign a score for it.
For the written examination, the field-test candidates sat for a six-hour assessment.
When Ms. Pence told the judges here of the length of time required for the written part of the process, there was a collective gasp in the university lecture hall.
But she said the laboratory expects to submit only a four-hour examination to the national board. Also, not all of the exercises done by the field-test volunteers will be submitted.
The volunteers were asked either to read an article examining two sides of an issue and then write about the arguments being presented or to respond to a description of a classroom scenario.
The 'Thinking' in Teaching
For the assessment-center exercises, candidates will be evaluated for what Ms. Pence called the "thinking involved with the teaching.''
The developers considered but discarded the idea of having the candidates actually teach a lesson to children they did not know, she said.
Instead, they will engage in a discussion about an educational problem with four of their peers. It will be videotaped for the judges.
Candidates will also be shown videos of a novice teacher in class and accompanying documents--lesson plans and the like--with which to make recommendations to the teacher on improving her performance.
In addition, the candidates will be given materials and asked to come up with suggestions for planning instruction taking into account students' language diversity. Ms. Pence said the exercise will not result in a traditional lesson plan.
The final exercise will ask the candidates to analyze students' writing.
Mr. Fones, the Zephyr Hills teacher, scored the discussion exercise. He said he came away from his three weeks here satisfied that the national board has drawn on a solid research base about good teaching to devise its assessments.
"My idea about what constitutes a good discussion has changed,'' he said.
There were times, watching the videotapes, when Mr. Fones said he "wasn't as impressed with a certain teacher, but at the same time I recognized that what the teacher was doing was what I'd do in the classroom. So I made a mental note.''
"My standards have been raised by participating in this process,'' he said.
The most problematic aspect of the portfolio exercises appeared to be the scoring dimension that evaluates a teacher's awareness of students' cultures.
Some of the judges here complained over lunch one day that such a multicultural emphasis only serves to underscore the differences between students and to exacerbate tensions in schools.
"That particular dimension is not one that is readily understood by most teachers, period,'' said Jamie Whitfield, a judge who teaches in Jacksonville, Fla. "But I think it needs to be there.''
Ms. Whitfield said teachers need "training across the board,'' beginning in college, so that they can address cultural issues adequately.
The dimension caused some confusion among the judges and needs to be clarified by the national board, she added.
"I'm hoping that this will push districts into saying, 'What exactly is this, and how can our teachers know that they are doing it correctly?''' she said of board certification.
Another longstanding concern about alternative types of assessment has been the technologies they use.
Both Mr. Fones and Charry Burr, a language-arts teacher in Philadelphia, said they quickly grew accustomed to watching the videotapes and, based on the video and the teachers' written commentaries, were able to make judgments about the teachers they observed.
At first, Ms. Burr said, "I didn't feel I would be able to watch 20 minutes of a teacher on a video and be able to see things for myself, but in five minutes I could tell.''
The judging process itself, she added, went surprisingly smoothly.
"I was surprised at how consistently the judges were just one point away from each other in working in the pairs,'' Ms. Burr said. "When they did not work in the pairs, they missed it.''
"They found that discussing the candidate really helped them,'' she said, "as they voiced their impressions, to come to an understanding that makes sense.''