School & District Management

Studies Examine States’ Test-Score Gaps

By Debra Viadero — June 20, 2001 5 min read
  • Save to favorites
  • Print

A pair of new studies spotlights racial and other differences cropping up in scores for state achievement tests in Massachusetts and Michigan.

It has long been recognized that the performance of African-American and Hispanic students tends to lag behind that of white and Asian-American students on most standardized tests, and the pattern in Michigan and Massachusetts has been no different. The new studies, however, take a closer look at those gaps and where they occur.

For the Massachusetts study, set to be published next month in the Harvard Educational Review, researchers compared 736 8th graders’ scores on the state tests with their grades in the same subjects. They found that the achievement gap between African-American and white students was bigger on the state mathematics tests than it was for classroom math grades.

To a lesser extent, the same pattern held for Hispanic students. Larger differences, though, turned up across gender lines, with girls outperforming boys in the classroom in math and science yet trailing behind them on state tests in the same subjects.

The Michigan study, in contrast, looked at questions from the state’s 11th grade test that had been made public and found 19 that were answered differently by black and white students who posted similar overall test scores. On one science item, in particular, the number of white students who gave the correct answer was 30 percent higher than it was for blacks. The study’s author theorizes that the disparities could have been enough to affect students’ eligibility for college scholarships.

“We want these tests to be as good as they can be,” said Ernest A. Bauer, the Michigan study’s author and a consultant for research, evaluation, and assessment for the Oakland Intermediate School District, which serves 28 suburban districts north of Detroit. “One has to be sure that what one is measuring is ability rather than some other demographic characteristic of the kid.”

Guarding Against Bias

In both states, education officials said they already take steps to ensure their tests are not skewed against any group of students. What the research may be picking up instead, they said, are differences in the classroom instruction that students in different demographic groups receive.

“If you look at any major standardized tests, you’re going to find differences along racial, gender, and economic lines,” said Stephanie G. Van Koevering, a spokesman for the Michigan Department of the Treasury, which oversees that state’s testing program. “In some cases, it’s not as much a flaw in the test itself as it is in the preparation of students.”

The question of potential test bias is important in both states because the tests carry high stakes for students. In Massachusetts, students finishing 10th grade this spring will have to pass the state tests, known as the Massachusetts Comprehensive Assessment System, or MCAS, in order to graduate. Test scores in Michigan, meanwhile, determine which students qualify for state-paid college scholarships of $2,500. Michigan testing officials, in fact, are already facing a lawsuit filed by the Michigan Civil Liberties Union, which argues that the test scores should not be the sole factor in distributing state scholarships.

Researchers in neither study could say whether test bias was to blame for the differences they found.

All that his analysis could show, said Robert T. Brennan, the Harvard University researcher who led the Massachusetts study, was that making MCAS scores the sole criterion for graduation could have a disparate impact on girls and some minority groups.

Because most minority students in Massachusetts attend poorer, inner-city schools, Mr. Brennan and his three co-investigators focused on four suburban school districts outside of Boston.

“When everyone is going to the same schools, you don’t have this problem of not knowing whether they’re doing worse on the tests because of where they go to school,” Mr. Brennan said.

The researchers found that, unlike the black and Hispanic 8th graders, Asian-American students’ grades were only slightly higher than their MCAS scores in almost every subject. Likewise, the margin by which girls outperform boys in the classroom in English is the same as it is on the state test in that subject.

It’s difficult to tell from the findings, however, which measure of a student’s academic ability is more accurate—teachers’ grades or state test scores, according to the researchers. Because no one can say for sure, they added, state officials should refrain from relying exclusively on state test scores to make critical decisions, such as the granting of high school diplomas.

Reflecting ‘What Is’

The top education official in Massachusetts said he was betting the state tests were the truer academic-achievement measure.

“The MCAS reflects what is; it isn’t causing the gap,” said Commissioner of Education David P. Driscoll. “We have any number of examples of young people who have gone through the school systems with passing grades and have gone on sometimes even to colleges and are shown to have a lack of skills.”

For his study on the Michigan Educational Assessment Program, Mr. Bauer also tries to level the playing field by focusing only on the 20,000 students across that state who scored right around the state- determined “cut score” for scholarships.

On the reading test, Mr. Bauer found five questions that white students were more likely to answer correctly and five that black students got right more often. The social studies test, which doesn’t count toward the scholarships, had only one question favoring whites, and none favoring blacks.

More troubling were the math and science tests, which had eight questions that whites were more likely to get right than blacks. Mr. Bauer stopped short of calling the test biased, however, because the questions that caused the most problems for black students did not seem to be outwardly skewed against them. The science question that showed the highest racial difference, for example, asked: “What formed the basins occupied by the Great Lakes?”

“You wonder how it would work if it just said, ‘What formed the Great Lakes?’” Mr. Bauer said.

Coverage of research is underwritten in part by a grant from the Spencer Foundation.

A version of this article appeared in the June 20, 2001 edition of Education Week as Studies Examine States’ Test-Score Gaps

Events

Special Education Webinar Reading, Dyslexia, and Equity: Best Practices for Addressing a Threefold Challenge
Learn about proven strategies for instruction and intervention that support students with dyslexia.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
The Key to Better Learning: Indoor Air Quality
Learn about the importance of improved indoor air quality in schools, and how to pick the right solutions for educators, students, and staff.
Content provided by Delos
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Leading Systemic Redesign: Strategies from the Field
Learn how your school community can work together to redesign the school system, reengineer instruction, & co-author personalized learning.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Litter Boxes in Schools: How a Disruptive and Demeaning Hoax Frustrated School Leaders
A hoax claiming that schools were providing litter boxes to students wasted school leaders' time as they worked to debunk it.
6 min read
Smartphone with blue and red colored hoax bubbles floating up off of the screen onto a dark black background with illegible lines of text also in the background.
iStock/Getty Images Plus
School & District Management A New Federal Grant Will Fund Schools' Energy Upgrades. Here's What to Know
The Department of Energy released new funding to help schools redo HVAC systems, add renewable energy, and upgrade facilities.
3 min read
A small white space heater directs air under a teacher's desk. On the front of the desk is a sign that says "Welcome to our classroom."
Personal space heaters are a common item found in the classrooms at Greene County High School in Snow Hill, N.C., where they're used to heat rooms when the HVAC units fail. New federal grants will help schools upgrade climate systems and add energy efficiency measures.
Alex Boerner for Education Week
School & District Management Opinion Principals, Make Room for Teacher Collaboration
The pandemic only reinforced the importance of dedicated time for professional development and collaboration.
Megan Stanton-Anderson
4 min read
112322 opinion Principal is IN 15Stanton Anderson collaboration
Vanessa Solis/Education Week via Canva
School & District Management 5 Stories of Inspiring Principals
For a dose of inspiration, look no further than these principals.
2 min read
Mike Huss, principal of Ione Elementary School in Ione, California, stands next to Jake, a former student who moved to North Carolina mid year.
Mike Huss, principal of Ione Elementary School in Ione, California, stands next to Jake, a former student who moved to North Carolina mid year.
Principal Mike Huss