The Educational Testing Service’s use of statistical methods to check for cheating on its college entrance examinations is fair, a New Jersey state judge ruled on Aug. 4. He also upheld the organization’s policy of invalidating the scores of students it suspects of cheating.
Superior Court Judge Richard S. Cohen ruled against four recently graduated Millburn (N.J.) High School students who had sued the testing organization, known as ETS, to prevent it from invalidating their scores on the Scholastic Aptitude Test (SAT) and for damages based on defamation of character, negligence, and infliction of emotional injury.
Judge Cohen, who did not attempt to determine whether the four high-school seniors cheated on the examination, wrote, “ETS’s approach to its necessary test security functions is sound and consistent not only with an appropriate evaluation of ETS’s relationships with its various clients but with the best interests of those clients.”
He added that “there were no departures from established ETS policies and procedures that unfairly affected the plaintiffs and the determination ETS made whether to question their scores.”
The case was the first full trial-court challenge to ETS’s procedures for dealing with suspected testing irregularities. The organization, in a prepared statement, called the decision “gratifying.” The students, who were graduated from the school this spring and have been accepted by colleges, have said they would appeal the decision. Judge Cohen barred ETS from informing the students’ colleges of the situation for 15 days from the date of his ruling.
Took Test Separately
As members of the Millburn High School tennis team, the four students were allowed to take the college-entrance test together under the supervision of their coach in May of 1982. Ten months later, they received form letters from ETS saying that a review panel had found an unusual amount of agreement among their answers. ETS initiated the review of the students’ scores after a receiving call from the students’ guidance counsellor, who had heard that they may have cheated.
The letter said the testing organization would cancel the students’ scores in two weeks and notify their prospective colleges, unless they could offer additional information to prove that they did not cheat. It also offered the options of appealing to impartial arbitrators, canceling their scores voluntarily, or taking the test again.
ETS customarily gives these options to the approximately 2,000 people who are suspected of cheating on the organization’s tests each year, according to an ETS spokesman. More than 5 million people took ETS examinations last year.
The students, claiming that they did not cheat on the SAT, sued instead of taking one of the firm’s options. They contended in court that ETS had not proven they cheated, that the organization should not be permitted to cancel test scores solely on the basis of statistics that suggest a likelihood of their cheating, and that the statistical models used in the analysis are unreliable.
Judge Cohen ruled in favor of ETS on each of these points. He noted that ETS’s use of the so-called “K-index,” which permits statistical analysis of the agreement of the incorrect answers of two test-takers, “is biased in favor of the [test-taker].”
However, George B. Gelman, the students’ lawyer, said in his remarks at the close of the three-week trial last month, “This is a case which invokes the abuse, the arrogance, and the corruption of power.” He called the K-index “a piece of statistical garbage.”
Of 85 questions on the verbal section of the SAT, all four of the Millburn students agreed on 42 correct answers, and three or more agreed on 17 incorrect answers, according to testimony. ETS experts saidthat such a high degree of three- and four-way agreement would occur less than one time in 300 billion by chance alone.
