CTB/McGraw-Hill knew of possible problems with one of its standardized tests several months earlier than the company previously acknowledged and eight months before its customers were notified, school officials in Indiana said last week.
Indiana was one of six states affected by the errors that skewed percentile rankings for students who took the company’s TerraNova battery of tests--and that caused thousands of New York City students to be mistakenly assigned to summer school or held back a grade.
McGraw-Hill, the publishing company that owns CTB/McGraw Hill, said last month that the testmaker first learned of the irregularities late last spring when testing directors in New York City and Tennessee voiced concerns. (“Error Affects Test Results in Six States,” Sept. 29, 1999.)
But district administrators in Indiana provided documents last week showing that they had raised questions about the scores as far back as January. The documents show that the testing company investigated the tests, corrected some errors, and then assured state officials and local superintendents that the results were sound--words the company had to take back in September when the new problems were made public.
“As far as I’m concerned, they [CTB officials] lack integrity,” said Thomas Fowler-Finn, the superintendent of the Fort Wayne schools and one of several local officials who questioned the results early on. “Even if they correct their errors, if they’re not dealing with us honestly it’s a real concern.”
David M. Taggart, the president of Monterey, Calif.-based CTB, said the company was not trying to cover up its mistakes when it assured Indiana administrators that all was well.
Though CTB officials identified a problem and corrected it, they did not probe deep enough, he explained.
“We did an analysis and came to some conclusions, but we did not discover at the time that anything was wrong with the calibration,” Mr. Taggart said last week. “Boy, I wish we had found this back then,” he added. “It would have made everything easier for everyone, and I do apologize to the people of Indiana.”
The glitches made public last month affected the percentile rankings, but not the raw scores, of students taking the TerraNova tests in Nevada, South Carolina, and Wisconsin as well as in Indiana and New York City. Officials in Tennessee caught the mistakes before results were sent to schools and students there.
The mistakes stemmed from a data-processing error that occurred when company researchers were translating student scores to percentile rankings, which show how students stack up national “norms"--meaning their peers from other states.
Protests to State
The problems Mr. Fowler-Finn of Fort Wayne and others identified in January also revolved around the norm-referenced results.
By Feb. 17, the company had analyzed the numbers and concluded that its researchers had used the wrong norming table for the reading-comprehension portion of the test. In a report to state officials, CTB said the mistake affected rankings only for that section of the test, in grades 6, 8, and 10.
But Mr. Fowler-Finn, an outspoken critic of Indiana’s testing program, and his district’s testing director, John N. Kline, were still dissatisfied. In letters and calls to state officials, they protested that the 3rd grade results continued to look suspicious. Their calls in March for an independent investigation went unanswered.
“He was saying things, and CTB was saying things, and there wasn’t any reason to doubt CTB at that point,” said Stewart Huffman, a communications specialist for the Indiana education department. The company had, for example, already rescored the tests at its own expense and mailed out new results.
Mr. Taggart said CTB did not take a second look at the scores until hearing from New York and Tennessee testing officials in May or June.
Audit Under Way
But angry Indiana officials said last week that the test-maker should have notified them and other customers by at least that point.
“I think it would have been bad enough had the error been brought to our attention immediately,” said David O. Dickson, a member of the Indiana board of education. “But it was compounded by the fact that it was not brought to our attention promptly, and we had to read about it on the front page of the paper.”
Company officials say it took several more months to pinpoint the cause of the problem. They notified the Indiana state superintendent, Suellen Reed, of their findings on Sept. 10.
