California education officials are working to steer their assessment and accountability system through stormy seas once again, after revealing that scoring errors on tests taken more than a year ago led the state to distribute some $750,000 in improvement awards to schools and employees not eligible to receive them.
The scoring glitch, which affected the 2000 scores for six districts and the 2001 scores for two districts, also means that staff members in 16 schools originally slated to receive bonuses of $5,000 to $25,000 will go home empty- handed when the checks are cut in the coming weeks.
Harcourt Educational Measurement, the San Antonio-based company that produces and administers the Stanford Achievement Test-9th Edition for California, discovered the mistakes in August after officials in one district called to clarify confusing patterns in the test results.
State officials say that while they are still sorting out the financial implications of the misdirected money, the individual teachers and staff members who received $580 bonuses last spring will not be expected to give the money back. Still, administrators in the six districts affected by the scoring mishap say its implications extend well beyond monetary concerns.
“We were so excited to learn that all of our schools made their targets on the tests, and now we find that’s not the case,” said Jean Fetterhoff, the superintendent of 8,500-student Kings Canyon school district near Fresno. “It’s just a downer for us.”
For staff members in the 16 schools that seemed to be in line to receive bonuses of $5,000, $10,000, or $25,000, the recent turn of events has been hard to swallow, officials said.
At the 900-student Kerman High School, also near Fresno, staff members had their eyes on the $25,000 prize after learning that the school had ranked fourth on the list of most-improved schools in the state. Under what the state calls the “Certificated Staff Performance Incentive” program—one of three bonus programs in a $677 million rewards package tied to the results of state tests taken last year—the certified employees in schools that made the biggest gains on state tests stand to receive the largest bonuses.
About 1,000 certified staff members in the most-improved schools are slated to receive $25,000 bonuses, 3,750 are to get $10,000 bonuses, and 7,500 are in line for bonuses of $5,000.
The state released a preliminary list of eligible schools in February that included Kerman and 15 other schools no longer in line for the bonuses. The official list of schools that will receive bonuses was slated to be released this week, with districts receiving the money later this month.
“It’s a big letdown,” said Scott Bishop, a math teacher at Kerman High School and the president of the Kerman affiliate of the California Teachers Association. “I was thinking about a down payment on a house or other investments, and I know other teachers were thinking about paying off their student loans.”
Earlier Problems
California’s bonus programs have seen their share of problems in the past. Earlier this year, the state delayed distribution of $577 million worth of bonuses to schools and their employees after officials learned that many students had opted out of taking state tests in a handful of schools in line to receive the performance awards.
Distribution of the remaining $100 million was further delayed by a lawsuit filed by a group of teachers at a Sacramento elementary school. The suit challenged the state’s requirement that schools log at least two consecutive years of improvement on state tests to be eligible for the rewards. (“Calif. Test-Based Bonus Plan Gets off to Rocky Start,” Jan. 24, 2001.)
Still, Ms. Fetterhoff of the Kings Canyon district said she was “just as happy” that the teachers in one middle school in her district were no longer eligible to receive the hefty bonuses. “You realize that people in one part of our district are working hard and have students that test well, while teachers in another part of our district are also working hard and have students that don’t test as well,” she said. “It was causing some morale difficulties.”
Harcourt, the testing company, attributes the scoring irregularities to a “human error” during the scoring process for both the 2000 and 2001 state tests. It says the tests of some 19,000 students were mistakenly scored as if they were taken in the middle of the school year, rather than at the end, when the scoring standards are more stringent.
The company was alerted to the problem when officials in the Kerman Unified School District noticed some apparent discrepancies in their 2001 test results, compared with their 2000 scores. Harcourt officials identified the error upon examining Kerman’s scores, then checked for similar irregularities in test scores throughout the state.
The contractor identified five other districts where scoring was similarly flawed in 2000 and two districts that received faulty scores this year. The 2001 scoring errors have no impact on the state rewards program because the state has yet to calculate who is eligible for bonuses based on those results.
Jean Shimko, the vice president of quality assurance for Harcourt, said that the company takes full responsibility for the scoring problem. But she noted that it affected only a tiny fraction of the state’s 6.2 million public school students.
“I’m not trying to make light of it,” Ms. Shimko said. “I think any error we have is a serious error. We take such great pride in quality work and making sure every student has an accurate score.”
Harcourt has since corrected the faulty data and sent new score reports to the affected schools. It is working with the state to resolve such issues as who will pay for the wrongly distributed reward money. In addition, Ms. Shimko said, the company has added a check into its scoring procedure to prevent similar errors from recurring.
Recovery Uncertain
As a result of the scoring errors, the state misdirected $750,000 in bonus money to six schools through two bonus programs last spring. The schools received roughly $525,000, while their employees got checks of $580 each that totaled about $225,000.
Doug Stone, a spokesman for the California education department, said the state does not plan to pursue money given to individual employees. Still, he said, “the state does not have a giveaway program where we could simply walk away from the problem.”
Mr. Stone said the department was negotiating with Harcourt to try to recover “some, if not all,” of the $525,000 that went directly to schools. “We’re trying to come up with a solution that creates as little discord to the schools as possible,” Mr. Stone said.
Wayne Johnson, the president of the California Teachers Association, argued that the recent mix-up only highlights the problems inherent in a system that hands out money to teachers and schools on the basis of test scores.
“This thing has created more anger, from people who didn’t get money, from people who didn’t get money and were told they were getting money, and from people who got money and didn’t want it,” Mr. Johnson said. “It’s created a lot of ill will.”
State Superintendent of Public Instruction Delaine Eastin sent a letter to districts last month acknowledging that the discovery of the scoring errors had “undercut our confidence in the integrity of our assessment and accountability systems.” But Ms. Eastin urged district leaders not to lose confidence in the overall strength of the state’s testing and accountability program.
“Of course, the damage cannot be undone,” Ms. Eastin wrote. “I remain convinced, however, that our accountability system is sound and plays a critical role in the improvement of our system.”
