Just as students were gearing up to take out their No. 2 pencils, Georgia officials decided to suspend most state tests in grades 1-8 this year after discovering that some 270 test questions were available on an Internet site for students, parents, and teachers.
All elementary and middle school students were to have taken the Criterion-Referenced Competency Tests starting this week. Instead, the state will give the tests only in grades 4, 6, and 8 in reading, mathematics, and English/language arts. Those are the only grades and subjects for which there is no evidence that students and teachers had access to secure test items within a state “item bank” that districts use for practice.
Kathy Cox, the state superintendent of schools, said the last-minute adjustments were “incredibly disappointing,” given how hard teachers and students had been working to prepare for the exams. “It really boiled down to a moral or ethical issue,” she said last week.
“How could I ask students and teachers to go through [testing], knowing that the results would come back and we wouldn’t be able to use them?” she said. “I just can’t see us putting people through the motions.”
The testing glitch is the latest in a string of such mishaps across the country that educators fear will only get worse, as states gear up to test hundreds of thousands of students under the “No Child Left Behind” Act of 2001.
The federal law, a reauthorization of the Elementary and Secondary Education Act, requires states to test every student in grades 3-8 and at least once in high school in English and mathematics by the 2005-06 school year. In the past few years, several states, including Georgia, have imposed steep penalties on testing companies for errors in administering or scoring statewide exams. (“Errors on Tests in Nevada and Georgia Cost Publisher Harcourt,” Sept. 4, 2002.)
“I think we are going to see these kinds of problems, simply because the demand is exceeding the capacity,” said Mark D. Musick, the president of the Atlanta-based Southern Regional Education Board. “States are going to have to take quality-control steps that I suspect most have not.”
Added Theodor Rebarber, the president of AccountabilityWorks, a Washington-based nonprofit group that worked last year with the Education Leaders Council to draft model standards for state testing contracts: “Clearly, there needs to be a new approach to the way states interact with testing companies. I think the states are in an unfortunate situation of having to reinvent the wheel in terms of managing these testing companies in every contract in every state.”
Contract Axed
Georgia was planning to use the results from this year’s tests for the first time to grade schools and reward or intervene in them based on their performance. Student-promotion decisions in grades 3, 5, and 8 also are to be based on test scores beginning with the spring 2004 administration of the exams. State legislators are now debating whether to postpone the school accountability system for one year.
The state already had been scrambling to keep its tests on schedule. On Feb. 21, the state administrative-services department rescinded a six-year, $84 million contract with Riverside Publishing, a division of the Boston-based Houghton Mifflin Co., after a competitor protested that all bidders had not received fair and equal treatment.
The department ordered a rebidding of the contract, which had been approved by Superintendent Cox’s predecessor, Linda C. Schrenko.
Less than a week later, on Feb. 26, members of the testing staff in the state education department were reviewing the printed test forms when they spotted a small number of items that also appeared in the nonsecure portions of the Criterion-Referenced Competency Tests’ item bank.
On March 10, Riverside provided a preliminary report that led the education department to conclude the extent of item exposure was “unacceptably high in most grade and content areas.”
On March 27, Ms. Cox and Wanda Barrs, the chairwoman of the state board of education, announced plans to sign an “emergency procurement contract” with Riverside, for $4.7 million, to enable a scaled-back version of this spring’s assessment to proceed as planned.
Eddie Pollard, the principal of the 400-student Tyrone Elementary School, south of Atlanta, was among the school administrators expressing disappointment.
“I think the teachers were, too, because they’ve been working so hard, and then all of a sudden, two or three weeks before the test, we’re told we’re not going to take it except in 4th grade in elementary school,” he said. “I’ve talked to more of my colleagues in different parts of the state, and they seem to feel the same way.”
State officials explained last week that they saw few options. Ms. Barrs said she hoped that teachers would continue to use the online item bank this spring “for their own diagnostic purposes” and to work with students to prepare for resumed testing in grades 1-8 next year.
“We just did not feel that we could go through with this process with the idea that these were valid tests, since they were not,” she said.
Assessing Blame
The debacle has led to a flurry of allegations about who’s at fault. Collin Earnst, the director of media relations for Houghton Mifflin, said Riverside was given the test questions by the previous contractor, Measured Progress, based in Dover, N.H.
“So the questions were there prior to us being given the contract and prior to Kathy Cox’s administration,” he said. “If there were any problems with these items, if they were released on any practice tests prior to us getting the questions, that was obviously out of our control. It happened before we ever saw them.”
Stuart R. Kahl, the president of Measured Progress, retorted that “it’s premature for that statement.”
Measured Progress is being paid by the Georgia education department to review the remaining tests in grades 4, 6, and 8 and ensure that all the items are secure. The company will also keep the online item bank, being used by tens of thousands of teachers and students daily, operating through the end of this school year.
Measured Progress created the bank, which contains three levels of questions: Web-based practice tests that parents and students can access from home; nonsecure items for school use; and a secure item bank for high-stakes, statewide exams.
Company officials say they prepared a database of all test items available for the 2003 CRCT that coded those used for the practice tests or teacher-item banking system, and thus were not available to be used on the secure statewide exams. They also identified questions that had appeared on previous years’ editions of the tests and were subsequently released to the public.
“To be quite honest, I don’t know if we’ll ever know exactly what happened,” said Ms. Cox, a Republican who was elected last November and took office Jan. 13.
The imbroglio also raises questions about the resources and capacity within the Georgia education department to oversee state testing contracts.
In 2002, Georgia officials decided not to pay Harcourt Educational Measurement for that year’s administration of the Stanford Achievement Test-9th Edition, which it had been giving in addition to the CRCT, after concluding that the results were no longer worth trying to salvage because of scoring problems.
The state board of education already had decided not to renew its contract with Harcourt. In 2001, the San Antonio-based company had been late in delivering the scores from that spring’s administration of the test, prompting state officials to insert penalty clauses into the 2002 contract.
“All of us have got to do a better job of staffing the state-level testing departments,” said Lisa Graham Keegan, a former Arizona state superintendent who is now the chief executive officer of the Washington-based Education Leaders Council. The ELC has held a number of meetings between representatives of states and test publishers.
“This stuff may have gone on before,” Ms. Keegan said, “but gone unnoticed because we just didn’t pay close enough attention and the stakes weren’t as high.”
“Rather than pointing fingers,” she added, “I think all of us need to decide this is a much bigger part of what our life is going to be like, particularly at the state level.”
Georgia gave the state curriculum tests in grades 4, 6, and 8 in 2000 and 2001. The CRCT exams were given in grades 1-8 for the first time last spring. Scores from this spring would have given educators a second year of data to measure progress.
Ms. Cox said she does not think Georgia has lost money as a result of the current testing problem. About $6.5 million of Riverside’s six-year contract was paid to the company through January of this year. The state paid the company another $1.1 million in February.
The state superintendent said she is now exploring whether more test development, administration, and scoring could be done in-house, working with local universities that might be more responsive to state needs.
“States have got to start looking to themselves,” she argued. “I think states are going to have to look to something different than these three or four testing companies.”
Ms. Cox and Gov. Sonny Perdue, a Republican, have asked the state legislature to delay the grading system for schools by one year. The GOP-controlled Senate approved the legislation. But the House, with a majority of Democrats, had not done so by late last week.
The state already had been considering a delay in order to reconcile its accountability system with the revised ESEA.
The state school board also is working on guidelines to promote or retain students in key grades based on data beyond state test scores.
In an unrelated move, board members voted last month to postpone high school end-of-course exams for a year. Ms. Barrs said they wanted to ensure that the tests are well aligned with state content standards and curricula.
“We just didn’t feel that the tests were ready for administration and implementation,” the board chairwoman said.
