Overruns Spur Teacher Board To Alter Plans
Faced with severe cost overruns in conducting its assessments, the National Board for Professional Teaching Standards has scaled back its time line for offering certification to accomplished teachers.
The organization has assembled a team of experts to offer suggestions on how to make the detailed evaluations of each applicant's work less expensive. The board also has canceled three of its seven contracts for developing assessments and will concentrate instead on retooling certificates now being offered.
The cost of evaluating the first crop of teachers was $4,000 per candidate, more than four times the $975 now being charged. That amount includes only the cost of operating the certification system, not the cost of developing the assessment materials.
Officials of the board, a privately organized group formed in 1987 to set standards for accomplished teaching and certify teachers who meet them, believe they can rein in costs without sacrificing quality.
"We are already confident we can achieve very substantial reductions in that initial cost," said James A. Kelly, the president of the Detroit-based board. "It's a very steep learning curve, and we're proud of the way we're climbing back up it."
The certification process includes portfolios that teachers compile from their work in the classroom, with videotapes of their teaching and samples of students' work. (See Education Week, 4/20/94.)
In addition, candidates visit an assessment center for interviews and written examinations.
From the start, educators and testing experts have questioned the feasibility of creating performance assessments that would meet testing standards and still be affordable.
Lee Shulman, the chairman of the national board's "rethinking task force," said he believes the board has proved that teaching can be evaluated with this type of assessment. The task now is to figure out how to do it less expensively, said Mr. Shulman, a professor of education and psychology at Stanford University.
"The challenge is to say, 'How far can we now begin to reduce the complexity of the assessment process, without sacrificing so much of the qualities we're striving for that we'll be back to where we started?'" said Mr. Shulman, whose research in the 1980's laid the groundwork for the national board's assessments.
Most of the expense for the first round of applicants went to scoring each candidate's work, an exhaustive process that in some cases took 23 hours.
"It is in some ways more remarkable that any assessments have been launched than it is that there are reasons to have to rethink," said Mari Pearlman, a task-force member who is devising mathematics assessments at the Educational Testing Service for the national board. "Every single piece of this assessment is new. We're not doing anything that we know from past experience will probably work."
In January, the organization announced that 81 teachers would become the first to receive national certification, out of 289 candidates.
The teachers in the first group are subject-matter generalists who work with children in early adolescence. Certifications will be awarded this summer to successful candidates from another group, made up of English-language-arts teachers who work with children of the same age.
Both groups underwent the review process last year, at no cost, as part of a field-test of the assessments.
This school year, the two assessments were offered on a nationwide basis for interested teachers, who paid a fee of $975. About 200 candidates went through the process.
Next year, however, neither assessment will be offered. They will be redesigned, then reintroduced in the 1996-97 school year.
The modifications will attempt to make the assessments less burdensome for both the scorers who review them and the teacher-applicants who must prepare them. Candidates in the first two field-tests, for example, reported spending about 100 hours each compiling the portfolios alone.
About three-quarters of the cost of administering the assessments, Mr. Kelly said, went to scoring. The teachers who scored the work, at a rate of $125 a day, first had to be trained.
Each section of the assessment was scored by two teachers, then rescored to check for reliability. Scoring each candidate's English-language-arts assessment, for example, required 14 assessors.
Because of problems with the original plan for scoring the assessment in English-language arts, the national board also had to revamp the scoring system for that certificate. That redesign proved both costly and time consuming, causing lengthy delays in the announcement of the results.
The double-scoring system, though expensive, is necessary to insure quality and reliability, officials say, in an assessment process based on human judgment, not machines.
The organization, which has a budget this year of $14 million, spent $2.5 million on a network of 26 field-test sites. Those sites recruited the first candidates, offered them support, and offered feedback on the assessments.
Using data from the field-tests, a separate technical-analysis group studied the assessments to make sure they met testing standards.
In the future, the developers of each assessment will do their own, shorter pilot tests. Technical analyses will use data generated as teachers go through the actual assessment process.
James R. Smith, the board's senior vice president, said the assessments asked for more material than was actually needed to make a judgment. The massive portfolios burdened both candidates and scorers, he added.
In rethinking the assessments, Mr. Smith said, "the main idea is to prune down these exercises so they require less work on the part of the candidate--less busywork--and are a lot more focused."
The national board in April notified three developers that it was canceling their contracts to produce new certification assessments.
Mr. Kelly said the board needed to spend its money retooling the existing assessments. The decision means that six certificates will be available by fall 1996. At one time, nine were in the pipeline.
The organization's $14 million budget this year is about the same as last year, with no fall-off in donations, Mr. Kelly said.
To date, the national board has raised $37 million from private donors and foundations and has received $19.34 million in one-to-one matching funds from the federal government.
But there was not enough in the budget to both retool assessments and create new ones, especially with the costs so far out of line with the candidates' fees.
In the future, the 63-member board of directors that governs the organization may decide to raise the fee, but no decision has been made, Mr. Kelly said. The fee already is seen as high for members of a relatively low-paying profession to pay. In some cases, states and districts pick up the tab.
Board members, two-thirds of whom are classroom teachers, supported the staff members' recommendations for rethinking the work.
"We're disappointed to have to slow down, but we also think it's the right thing to do," said Joyce Ojibway Jennings, a board member who chairs the working group on assessments. "A great deal of care has been taken to get it right and not just push ahead at all costs."
Gary R. Galluzzo, the dean of the college of education at the University of Northern Colorado and a new member of the board, said the assessments are worth the wait.
"They appear to be capturing the complexity of teaching and are respectful of teachers," he said. "We're actually tapping expertise here."
The canceled contracts were with:
Steve Schneider, an assessment developer at Far West Laboratories, said people there "were all a little shocked" by the cancellation of the contract.
"This is fairly rare," he said. "To be up and running and doing good work, and being reaffirmed from the field, and then having the financial rug pulled out is very difficult."
But Mr. Schneider and other assessment developers said they understood the board's decision to perfect its first assessments before spending more on new ones.
"We respect the fact that they have to take a pause, and we understand that the work needs to be reviewed thoughtfully and deliberately," said Carolyn Wyatt, who was directing the development lab at the Education Development Center. "People need time to be reflective about the process."
Emphasis on Standards
Through some refinements, the cost of some assessments already has fallen, Mr. Kelly said.
The cost for administering the assessments this year for the approximately 200 teachers in the fields of early adolescence/generalist and English-language arts, for example, dropped to about $3,000. The field-test for the early-childhood/generalist and middle-childhood/generalist certificates, now under way, will cost about $2,500.
In the fall, the certificates for generalists working with children in early and middle childhood will be offered on a nationwide basis.
The following year, the national board expects to offer those two certificates, revamped early-adolescence/generalist and early-adolescence/English-language-arts certificates, and assessments in adolescence and young adulthood/mathematics and early adolescence through young adulthood/art.
In addition, 17 committees are at work setting standards in 21 certification fields. The standards are used by the assessment-development laboratories to determine what qualities to look for in expert teachers.
Mr. Kelly said the national board plans to "make a much bigger deal" out of its standards in the coming months, forming partnerships with states, districts, and professional groups to use the standards to help practicing teachers enhance and improve their teaching.