The gender gap on standardized tests is real but usually not large, concludes a four-year study by the Educational Testing Service that also found more similarities than differences marking girls’ and boys’ performance.
The study by the makers of the SAT college-entrance exam, which has been a prime target of critics claiming bias against girls, was released here last week. It summarizes an upcoming book on the subject by researchers at the Princeton, N.J.-based testing service.
As other studies, including the 1996 science exam of the National Assessment of Educational Progress, have found, girls’ scores on math and science measures have dramatically improved in the past 30 years to narrow the gap between their performance and that of boys, the ETS study says. And the report reiterates that gender gaps on tests cut both ways: Boys show a continuing deficit compared with girls when it comes to writing and language skills.
What gender differences do exist on tests are complex and not easily explained by such factors as course-taking patterns or types of tests, according to the report, “The ETS Gender Study: How Females and Males Perform in Educational Settings.”
Instead, performance differences between boys and girls occur before their course-taking patterns begin to diverge, and they appear across a wide variety of tests and other measures, the study says. Gender differences are also reflected in the different out-of-school activities of boys and girls, the researchers note.
The study also found that gender differences were not affected by adding more time to tests or testing students under conditions in which guessing played less of a role. Also, the report found that in controlled studies that examined whether a question called for an open-ended response or a multiple-choice one, differences between the sexes were unaffected.
But the recent NAEP science exam, given to a nationally representative sample of more than 22,000 public and private school students, found that unlike on previous national science assessments, 8th grade boys did not significantly outperform girls in that grade. Experts theorize the improvement could be due, at least in part, to the inclusion of open-ended questions. The ets was unable to include the NAEP results in its study because they were released just this month. (“NAEP Assigns No Grades on Science Exam,” May 7, 1997.)
Smoke Screen?
The ETS researchers reviewed existing studies and looked at data from more than 400 different tests and other measures--going back 30 years in some cases--that reflect the performance of more than 15 million students. The report includes results both from tests taken by a nationally representative sample of students, such as NAEP, and those taken by a self-selected sample of college-bound students, such as the SAT I: Reasoning Test and the ACT.
In addition to the SAT and the Preliminary SAT, the nonprofit testing service administers and scores such tests as the Graduate Record Examinations.
Both the New York City-based College Board, which sponsors the SAT and PSAT, and the testing service have consistently denied charges that their tests discriminate against girls. However, girls’ SAT math scores last year lagged 34 points behind boys’, on average, on the 200-to-800-point scale. Boys scored 4 points higher than girls on the verbal section. (“SAT, ACT Scores Up, But Racial Gaps Remain,” Sept. 4, 1996.)
Critics argue that girls lose out when admissions to college and scholarship awards are based on such scores. And, they contend, because girls tend to get better grades than boys in high school and college, the SAT fails at its stated goal of accurately predicting college performance.
In the study, the ETS acknowledges that shortcoming but argues that grades and tests measure different types of skills and should be used together to make judgments about students.
“It’s laudable that ETS researches its own tests,” said Marcia C. Linn, a professor of education at the University of California, Berkeley. “Still, my question is, can they design a test that doesn’t underpredict [women’s college grades], and what would that test look like?”
One detractor blasted the study. “It’s a smoke screen designed to divert attention from the ongoing problems with the exams that they publish,” said Bob Schaeffer, the public education director at the National Center for Fair & Open Testing, or FairTest, a Cambridge, Mass.-based watchdog group. “What does it prove for a big company to publish a research paper that says its product is not flawed?” Mr. Schaeffer said.
‘Trying To Learn’
But Nancy S. Cole, the president of the ETS and one of the study’s authors, said that the study aims to shed light on conflicting claims about large and small gender differences in student performance and to help the testing service improve its tests.
“Our motivation for doing research is to find out where we need to change so we can change,” Ms. Cole said in an interview.
“We’re not absolving ourselves,” she said. “We’re just saying that the notion that people have that there are some differences that must be bias just didn’t hold up in the study.”
In its review of existing studies, the report does its best to refute some of the allegations leveled against ETS tests and repeatedly knocks down claims of bias.
One finding, which the testing center acknowledges is not new, has particular significance for how students do on the SAT, and the study’s conclusions about fairness rely heavily on it.
The study found that results from tests given to nationally representative samples of 12th graders reveal a bell-shaped distribution of scores for both genders, but with more males appearing at both the very lowest end and at the very highest.
As a result, when self-selected groups of high-achieving students take such tests as the SAT, the new report argues, the score distribution reflects only the male-dominated upper end of the spectrum of test-takers.
In addition, the report says, the skills tested on high-stakes tests such as the SAT may be those in which males perform better, such as mathematical reasoning and concepts, and the skills on which females excel, such as writing, “have either been overlooked or have been difficult and expensive to measure.”
Such results, the study points out, “are not surprising or an indication of bias but are expected and follow from the results in representative samples.”
Yet, the ets agreed last year to the settlement of a federal civil rights complaint by FairTest alleging discrimination against girls in the awarding of National Merit scholarships, which are based on PSAT scores. Under the agreement, the ETS promised to add a writing-skills test to the PSAT. (“PSAT To Add Writing Test To Settle Bias Case,” Oct. 9, 1996.)
