For the second time in as many weeks, a government contractor scoring results from “the nation’s report card” has announced that it made a mistake.
Last week, American College Testing sent a letter to the U.S. Department of Education acknowledging that it had made an error in computing the student-achievement levels for the 1992 math and reading tests for the National Assessment of Educational Progress, known as the nation’s report card. The error also affects the 1994 reading tests because the same achievement levels were used for that test.
The error by the act, based in Iowa City, Iowa, comes close behind that acknowledged by the Educational Testing Service of Princeton, N.J. Earlier last month, the ets revealed that a computer program botched the scoring for the results from the 1992 assessments in math and reading and the 1994 reading and world-geography tests. The impact of that mistake was said to be negligible. (See Education Week, Sept. 27, 1995.)
Officials familiar with the mistake by the ACT said that it, too, was fairly slight and would not change trend data or the rankings of states.
The ACT mistake arose from an error in the computer software used to compute the levels of student achievement--basic, proficient, or advanced--on NAEP, said Gary Phillips, an associate commissioner at the Education Department’s National Center for Education Statistics. In effect, the error overemphasized the importance of items that ask students to write a response. It did not affect multiple-choice items. NAEP is a project of the statistics center.
Damage Control
The result of fixing the mistake is a lowering in the scale score associated with a given achievement level. But as of late last week the National Assessment Governing Board, which sets the achievement levels, had not announced whether it would reset them.
In the next few weeks, the contractors are expected to correct the errors and provide the department with revised data. Mr. Phillips said the contractors, not the federal government, will pick up the costs associated with the errors.
Later this month or early next month the Education Department will reissue the “1994 First Look Report” with revised 1994 NAEP reading data, according to Jeanne Griffith, the acting commissioner of the statistics agency. The complete reports on national- and state-level reading performance from 1994, due out this fall, will be revised and released after the “First Look” report, she said.
Ms. Griffith called the mistakes troubling and said the agency regrets any problems it created for those who use NAEP results.
“It is something that we take extremely seriously,” she said of the errors. “As a statistics agency, we think it’s essential we stand by our data and why we must correct errors when we find them.”
“It is terribly unfortunate that this has happened, and we will do everything in our power to see what mechanisms can be used to reduce the likelihood that that happens again,” she added.
The mistakes mark the first time that results already made public have had to be corrected, officials said. Those familiar with the assessment point out that compiling results from the test is a very complex undertaking. The assessment governing board, which oversees NAEP, is looking into simplifying the task.
Ms. Griffith praised both testing services for coming forward promptly when they found the errors. Ets turned up the ACT mistake in reviewing its own error.
As part of an existing contract, the department has asked the accounting firm KPMG Peat Marwick to review the mistakes.
