The “imbalanced” results of teacher evaluations administered recently under Florida’s Master Teacher Program have prompted educators and lawmakers in that state to call for a moratorium on the entire merit-pay plan.
Last week, a coalition of groups representing teachers and administrators called for an immediate halt to the Master Teacher Program until a legislative inquiry can be conducted into the evaluation procedure that serves as one of two methods used to determine master-teacher status.
In addition, legislators last week intro-duced amendments to the appropriations bill that would cut off funds for the program and place the $20 million earmarked for merit pay into the state’s general fund for education. According to Representative Tommy Hazouri, chairman of the education committee, the funding issue will be resolved by a conference committee this week.
‘Defective’ Concept
The calls for a moratorium were provoked by the uneven results of the teacher evaluations, released on May 9. The proportion of teachers qualifying for the Master Teacher Program varied widely among districts and among schools, according to John Ryor, executive director of the Florida Teaching Profession, an affiliate of the National Education Association.
“This is what happens when political bodies rush to put something in place that, in our judgment, is defective as a concept,” Mr. Ryor said.
Both the nea and the state affiliate of the American Federation of Teachers have opposed the merit-pay plan from its inception.
One of the first such state efforts in the country, the Master Teacher Program was approved by the legislature in 1983 and put into effect this year. Teachers who qualify for merit pay are to receive bonuses of $3,000. The $20-million program provides for bonuses for up to 6,333 teachers, on the basis of two separate performance evaluations and their scores on subject-area examinations.
From the beginning, the program has been plagued with problems ranging from lawsuits filed by both unions to disagreements about the examination and evaluation methods to be used. (See Education Week, Feb. 20, 1985.)
24,000 Teachers Evaluated
Charles Sanders, an aide to Gov. Robert Graham, acknowledged a widespread concern among educators and lawmakers over the “distributional effects” of the evaluation.
In addition to passing the subject-matter test, teachers must obtain a score in the top quartile statewide on the evaluation to be considered for master-teacher status. But in some school systems an unlikely number of teachers ended up in the top 25 percent.
For example, Mr. Sanders explained, of the 31 teachers who were evaluated in Baker County, 20 scored in the top 25 percent; in Leon County, 151 out of 270 teachers were in the upper quartile.
And in other school systems, surprisingly few teachers scored high enough to be in the top 25 percent. “One had only four out of 50 make it,” Mr. Sanders reported, “and another had two out of 25.”
Mr. Sanders said the state’s Teacher of the Year did not score high enough, nor did one of the state finalists for the national Teacher in Space competition.
In addition, none of the 50 instructors evaluated at Terry Parker High School in Jacksonville scored in the top 25 percent on their evaluations. The U.S. Education Department honored Terry Parker High School last summer in its Secondary School Recognition Program.
About 24,000 of the state’s 90,000 elementary and secondary teachers were evaluated, according to state officials. Another 35,000 have taken the subject-matter tests. Under the Master Teacher Plan, which teachers participate in on a voluntary basis, they must pass their subject-area examination in addition to scoring in the top quartile on their evaluation in order to become an “associate master teacher.”
State officials expect about 2,000 to 3,000 teachers to qualify for the program this year, Representative Hazouri said.
Some Variation Expected
State education officials expected some variations from district to district in the percentages of teachers who obtained scores on their evaluations in the top quartile, said Ralph D. Turlington, Florida’s commissioner of education. “But,” he said, “we had some variations that were significantly greater than you could expect by statistical chance or, in my judgment, by differences in the personnel.”
Mr. Turlington “hypothesized” that the uneven distribution of teachers who scored high enough on the evaluation may be due to “some significant differences in the way the evaluations were applied or handled.”
The state education department will conduct its own analysis of the evaluation, he said. The analysis could include an examination of the evaluators and their training, the information and expectations teachers had about the process prior to their evaluations, and the evaluation instrument itself.
Effectiveness Indicators
The evaluation instrument, the “Florida Performance Measurement System,” is a one-page form with a list of 20 “effectiveness indicators’’ and 18 “ineffectiveness indicators.” On the form, the trained evaluator, who is an administrator in the school district, makes a mark every time the teacher exhibits positive or negative behavior.
For example, a teacher receives a mark of effectiveness if he begins class promptly, asks students questions, or provides students with reviews of the instructional material at the beginning and end of class.
The teacher is marked ineffective if he uses “vague or scrambled discourse,” is disorganized in the handling of instructional materials, or “poses nonacademic questions.”
Each teacher is evaluated at least twice by different evaluators and the evaluations are scored by machine.
Mr. Turlington said that there may be some “bugs” in the Master Teacher Program and that “given the time frame” the department of education had to implement the program, “it would not be unreasonable to anticipate some very serious implementation problems.”
“I think now we should take time to take stock,” he said. “When you are seeking to designate ‘special’ persons, you want to be as evenhanded as possible.”
Governor Continues Support
Despite questions raised by the uneven results of the evaluation, Mr. Sanders said that Governor Graham, who was instrumental in getting the Master Teacher Program through the legislature in 1983, still backs it.
“We went into this thing and we’ve worked on it hard for two years,’' Mr. Sanders said. “We think it’s a workable program or we wouldn’t be in it in the first place. This is just a bump in the road.”
The Governor is concerned, however, about the evaluation results and will ask for an analysis of the scoring, Mr. Sanders said.
Governor Graham also is worried about last week’s legislative proposals to restrict funds for the program, according to Mr. Sanders. “It’s too early to say what the outcome of this is going to be,” he said, “but it would be bad news if they killed the Master Teacher Program.’'
Mr. Sanders added that one problem with the evaluation system may be that many people do not understand how it works. “Not everybody is statistically literate,” he said.
He also suggested that the most recent opposition to the merit-pay plan may be political manuevering by the teachers’ groups who oppose merit pay in favor of across-the-board salary increases for all teachers.
“Maybe the problem is that the unions have fought this ever since it was first proposed,” he said.
‘Ill-Considered’ Evaluation
But according to Mr. Ryor of the state’s nea affiliate, the evaluation method “just isn’t working.”
“The evaluation instrument is a silly, ill-considered document to measure teacher success or failure,” Mr. Ryor said. It was developed hastily, he said, and the results demonstrate that it is not reliable.
The major problem with the program, Mr. Ryor said, is that its object “is not to identify excellent teachers, but to cut the number who qualify down to fit the available money.”
“I’m afraid we rushed into this,” added John Gaines, executive director of the Florida Association of School Administrators, which joined the two teachers’ unions, the Florida Legislative Alliance, and the Florida School Boards Association in calling for a moratorium on the program.
While the administrators “are not opposed to merit pay as a concept,” Mr. Gaines said, “we had questions about [the program] in the beginning, but nobody bothered to listen to us.”
A New Procedure
In an interview last week, Donovan Peterson, one of the developers of the evaluation instrument, defended the process but said that “if a moratorium will help us in the investigation of the imbalance of awards between schools and districts, to that degree I support it.”
“This is a new program that is being attempted on a statewide basis, and it must be treated as such,” said Mr. Peterson, a professor of education at the University of South Florida in Tampa. “We must give any problem inherent within the program our thorough attention to assure that we are being fair in treating all of the teachers with equity.’'
He said that the evaluation instrument was validated by a number of Florida educators and a team of “nationally known experts in the field of teacher effectiveness.”
Mr. Peterson also rebutted charges that the evaluation instrument was developed “hastily,” pointing out that he and other researchers had been working on it long before the state adopted the master-teacher program. “While we don’t consider the instrument beyond need for further investigation, there is substantial evidence of its reliability and validity,” Mr. Peterson said.
“As a group of independent researchers, we were concerned that the instruments used to evaluate teacher performance were of a general nature, lacked connections with the research literature on teacher effectiveness, and that the users in most cases received no training.”
Mr. Peterson said he “could not speculate on the reasons for the imbalance until we’ve had an opportunity to conduct further study.”
Reconsideration of Program
According to Representative Hazouri, a Democrat from Jacksonville, the evaluation results compel the legislature to “rethink” the program during the next three weeks.
“I know that we have excellent teachers here and that those who are in the upper quartile are excellent,” he said. “But they also are some of the luckiest. It’s easier to be a winner in the Reader’s Digest sweepstakes than to be a merit teacher.”
He predicted that the legislature will provide funds for the bonuses of the teachers who qualify for the program this year but that in its next session the legislature will “review alternatives, such as career ladders.”
“I think the biggest problem of all right now is morale,” he said. “This is not undoing education in Florida, but it has given an already overburdened teaching force an even greater burden.”
