Ky. Fires Firm That Ran Innovative Testing Program

By Millicent Lawton — July 09, 1997 3 min read

After a six-year partnership, Kentucky officials have fired the testing contractor responsible for a mistake that changes how all of the state’s elementary and middle school stacked up on last year’s test of student achievement.

The error also cost perhaps thousands of teachers, at least temporarily, the financial reward they were owed under the pioneering Kentucky system that links student performance and pay. And while boosters have voiced support for the testing program since the blunder became public, at least one state education official said that critics may try to use the error to their advantage in ongoing attacks on the controversial assessment.

Indeed, state legislators last week authorized a weeping audit covering the life of the testing collaboration, including management by the state education department. They also froze any remaining payments to the contractor, which could delay the scoring of this year’s test or the administration of next year’s exam.

Commissioner of Education Wilmer S. Cody made the announcement late last month that Advanced Systems in Measurement & Evaluation Inc., based in Dover, N.H., made the data processing error. At the same time, he said the state was terminating after one year its current four-year, $32 million contract with the firm.

The scores of the state’s more than 1,000 elementary and middle schools will increase slightly from those reported on the 1995-96 Kentucky Instructional Results Information System, or KURIS. Some of those schools--perhaps one-third--may move up one or more categories on the state’s performance rankings.

The affected scores are those of students in grade 4 and 8 who took tests in arts/humanities and practical living/vocational studies. The error occurred when an Advanced Systems employee used the wrong set of data to calculate scores on those two subject tests.

Kentucky makes cash rewards to teachers in school with strong performance on the tests. As a result of the error, the state may have to payout about $2 million in additional bonuses.

Assessing Damage

But the extent of the damage won’t be known for some time, said Ed Reidy, the director of assessment for Kentucky.

“Obviously, we’re not happy with an error of this sort, but we’re not cursing the darkness,” he said. “We’re dealing with it as forthrightly, rapidly, and, I hope, as thoroughly as we can.”

Advanced Systems officials said they agreed with Kentucky’s termination of their contract. “Were we in the same position, we would be making the same decision,” said Sharman Price, a spokeswoman for the company.

But Advanced Systems, which has revenue of about $16 million to $20 million a year, say it has now lost about 20 percent of its business. The company, which was founded in 1984 and has 84 employees, retains state-testing contracts in eight other states.

The high-stakes nature of the Kentucky test magnified the import of what Ms. Price called “subtle” errors. Officials at both the state education department and Advanced Systems suggested that the ax came down too swiftly on the contractor because of the political implications of potential rewards or sanctions for schools and their employees.

Questions remain about how the mistake could have gone undetected for many months. Officials at Advanced Systems pointed out that Kentucky officials had also failed to catch the error until last month.

State officials will work to keep the testing system as much on schedule as possible, Mr. Reidy said, but he acknowledged it will be difficult given the need to start over in finding a contractor to run the program.

Since it was hired by Kentucky in 1991, Advanced Systems has helped the state create a cutting-edge assessment of what students can show about what they know, rather than asking them to regurgitate facts on a paper-and-pencil test. (See Education Week, June 8, 1994.)

Last year, the company discovered that it had used an incorrect equation to calculate scores on a Maine assessment. Ms. Price said that the two mistakes were isolated incidents.

A version of this article appeared in the July 09, 1997 edition of Education Week