ETS Disputes Charges of Gender Bias

By Millicent Lawton — May 14, 1997 5 min read


The gender gap on standardized tests is real but usually not large, concludes a four-year study by the Educational Testing Service that also found more similarities than differences marking girls’ and boys’ performance.

The study by the makers of the SAT college-entrance exam, which has been a prime target of critics claiming bias against girls, was released here last week. It summarizes an upcoming book on the subject by researchers at the Princeton, N.J.-based testing service.

As other studies, including the 1996 science exam of the National Assessment of Educational Progress, have found, girls’ scores on math and science measures have dramatically improved in the past 30 years to narrow the gap between their performance and that of boys, the ETS study says. And the report reiterates that gender gaps on tests cut both ways: Boys show a continuing deficit compared with girls when it comes to writing and language skills.

What gender differences do exist on tests are complex and not easily explained by such factors as course-taking patterns or types of tests, according to the report, “The ETS Gender Study: How Females and Males Perform in Educational Settings.”

Instead, performance differences between boys and girls occur before their course-taking patterns begin to diverge, and they appear across a wide variety of tests and other measures, the study says. Gender differences are also reflected in the different out-of-school activities of boys and girls, the researchers note.

The study also found that gender differences were not affected by adding more time to tests or testing students under conditions in which guessing played less of a role. Also, the report found that in controlled studies that examined whether a question called for an open-ended response or a multiple-choice one, differences between the sexes were unaffected.

But the recent NAEP science exam, given to a nationally representative sample of more than 22,000 public and private school students, found that unlike on previous national science assessments, 8th grade boys did not significantly outperform girls in that grade. Experts theorize the improvement could be due, at least in part, to the inclusion of open-ended questions. The ets was unable to include the NAEP results in its study because they were released just this month. (“NAEP Assigns No Grades on Science Exam,” May 7, 1997.)

Smoke Screen?

The ETS researchers reviewed existing studies and looked at data from more than 400 different tests and other measures--going back 30 years in some cases--that reflect the performance of more than 15 million students. The report includes results both from tests taken by a nationally representative sample of students, such as NAEP, and those taken by a self-selected sample of college-bound students, such as the SAT I: Reasoning Test and the ACT.

In addition to the SAT and the Preliminary SAT, the nonprofit testing service administers and scores such tests as the Graduate Record Examinations.

Both the New York City-based College Board, which sponsors the SAT and PSAT, and the testing service have consistently denied charges that their tests discriminate against girls. However, girls’ SAT math scores last year lagged 34 points behind boys’, on average, on the 200-to-800-point scale. Boys scored 4 points higher than girls on the verbal section. (“SAT, ACT Scores Up, But Racial Gaps Remain,” Sept. 4, 1996.)

Critics argue that girls lose out when admissions to college and scholarship awards are based on such scores. And, they contend, because girls tend to get better grades than boys in high school and college, the SAT fails at its stated goal of accurately predicting college performance.

In the study, the ETS acknowledges that shortcoming but argues that grades and tests measure different types of skills and should be used together to make judgments about students.

“It’s laudable that ETS researches its own tests,” said Marcia C. Linn, a professor of education at the University of California, Berkeley. “Still, my question is, can they design a test that doesn’t underpredict [women’s college grades], and what would that test look like?”

One detractor blasted the study. “It’s a smoke screen designed to divert attention from the ongoing problems with the exams that they publish,” said Bob Schaeffer, the public education director at the National Center for Fair & Open Testing, or FairTest, a Cambridge, Mass.-based watchdog group. “What does it prove for a big company to publish a research paper that says its product is not flawed?” Mr. Schaeffer said.

‘Trying To Learn’

But Nancy S. Cole, the president of the ETS and one of the study’s authors, said that the study aims to shed light on conflicting claims about large and small gender differences in student performance and to help the testing service improve its tests.

“Our motivation for doing research is to find out where we need to change so we can change,” Ms. Cole said in an interview.

“We’re not absolving ourselves,” she said. “We’re just saying that the notion that people have that there are some differences that must be bias just didn’t hold up in the study.”

In its review of existing studies, the report does its best to refute some of the allegations leveled against ETS tests and repeatedly knocks down claims of bias.

One finding, which the testing center acknowledges is not new, has particular significance for how students do on the SAT, and the study’s conclusions about fairness rely heavily on it.

The study found that results from tests given to nationally representative samples of 12th graders reveal a bell-shaped distribution of scores for both genders, but with more males appearing at both the very lowest end and at the very highest.

As a result, when self-selected groups of high-achieving students take such tests as the SAT, the new report argues, the score distribution reflects only the male-dominated upper end of the spectrum of test-takers.

In addition, the report says, the skills tested on high-stakes tests such as the SAT may be those in which males perform better, such as mathematical reasoning and concepts, and the skills on which females excel, such as writing, “have either been overlooked or have been difficult and expensive to measure.”

Such results, the study points out, “are not surprising or an indication of bias but are expected and follow from the results in representative samples.”

Yet, the ets agreed last year to the settlement of a federal civil rights complaint by FairTest alleging discrimination against girls in the awarding of National Merit scholarships, which are based on PSAT scores. Under the agreement, the ETS promised to add a writing-skills test to the PSAT. (“PSAT To Add Writing Test To Settle Bias Case,” Oct. 9, 1996.)

Related Tags:


Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Engaging Young Students to Accelerate Math Learning
Join learning scientists and inspiring district leaders, for a timely panel discussion addressing a school district’s approach to doubling and tripling Math gains during Covid. What started as a goal to address learning gaps in
Content provided by Age of Learning & Digital Promise, Harlingen CISD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
How to Power Your Curriculum With Digital Books
Register for this can’t miss session looking at best practices for utilizing digital books to support their curriculum.
Content provided by OverDrive

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
Facing the Future Together: Digital Innovative Solutions
Join us to discuss how digital innovative solutions can enrich the educational experience in the K-12 environment. We’ll share how these ...
Content provided by Pearson
Assessment Opinion What Federally Mandated State Tests Are Good For (And What They Aren’t)
Spring 2021 testing is happening. That can be a good thing—if the goal is about more than school accountability.
Stuart Kahl
5 min read
Two people analyze test data
Visual Generation/iStock/Getty
Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
How Are Kids Really Performing This Year?
Has the academic impact of COVID-19 narrowed since last fall? How much progress have students made at each grade level? Get the answers i...
Content provided by Renaissance Learning