Student Achievement

Testing Experts Develop New Method of Presenting Achievement-Gap Data

By Lynn Olson — March 13, 2002 3 min read

What’s the gap in performance between two runners at the end of a marathon? The answer can be expressed in minutes, or more likely fractions of seconds, to the finish line.

But how do you describe the gap between groups of runners, such as men and women? Do you look at the differences in their average finishing times, or focus on the slowest or the fastest runners in each group? That’s a more complicated problem, but similar to determining whether states are closing the achievement gaps between groups of students, such as those of different races or family-income levels.

A report prepared by the board that governs the National Assessment of Educational Progress cautions that measuring an achievement gap does not come down to a single statistic. Comparing just the average scores for two groups of students, or the percent above the “basic” level on state or national tests, could miss some important changes, the report warns, and misrepresent what’s happening.

For example, while the percentage of students who meet or exceed a standard may be unchanged from one year to the next, students’ scores below the standard may have risen substantially.

“The key idea is that, in some way or another, we must compare whole groups of scores,” said Paul W. Holland, who holds a chair in measurement and statistics at the Princeton, N.J.-based Educational Testing Service. He was a member of a working group that helped prepare the board’s report on using NAEP to confirm state test results.

To help visualize changes in test scores and test-score gaps for whole groups of students over time, Mr. Holland turned to graphing methods often employed in such fields as medical research, insurance, and engineering. Known as “cumulative-distribution functions,” such charts can display test scores across the entire range of performance simultaneously, thus making progress relatively easy to spot.

In the first chart above, for example, the lavender and orange curves show the percent of students who performed below each NAEP score in 4th grade mathematics for two groups of students from State A: 4th graders who took the test in 1996 and were eligible for the federal free-lunch program, and 4th graders who took the test in 1996 and were ineligible for the program, respectively. By looking at where the curves intersect the line marked “basic,” you can see that in 1996, more than half the students from low-income families, or those eligible for free lunches, scored below the basic level on the test, compared with about 23 percent of their better-off counterparts.

It’s also clear from the chart that, four years later, the performance of both groups improved across the board because the curves have both shifted to the right.

In 2000, for example, about 40 percent of 4th graders from low-income families performed below basic on the test, compared with about 15 percent of their more affluent peers. If there were no space between the curves for the two groups, it would mean that 4th graders eligible and ineligible for free lunches performed equally well. As the National Assessment Governing Board report presented here notes, “This is a completely new way of representing ‘gap’ in achievement on NAEP.”

‘Truth in Gaps’

The second chart zeroes in on the actual size of the achievement gap between the two groups and whether it has closed over time. It shows that a gap exists at almost every point along the continuum, but that the gap was higher in 1996 than in 2000. Fourth graders eligible for free lunches who scored at the 30th percentile in 1996, for example, had NAEP scale scores that were 26 points lower than those of their better-off peers. Four years later, that difference had dropped to 20 points.

Taken together, the two charts show that while both groups improved, students eligible for the free- lunch program have “gained ground,” and the narrowing of the gap should be considered real.

“To me, it’s like truth in gaps,” Mr. Holland said. “If you just look at one place along the scale—which is what you do if you look at achievement levels—you don’t see anything else. And these distributions typically are changing in a variety of ways, including places where the achievement levels are not set.”

The governing board is considering whether all future NAEP reports should include such kinds of information.

“This gives us so much more information than any chart we’ve had in the past,” said Marilyn A. Whirry, a governing-board member and a high school English teacher from Manhattan Beach, Calif., “and it’s more honest information.”

A version of this article appeared in the March 13, 2002 edition of Education Week as Testing Experts Develop New Method of Presenting Achievement-Gap Data

Events

School & District Management Live Event EdWeek Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Whitepaper
How Are Kids Really Performing This Year?
Has the academic impact of COVID-19 narrowed since last fall? How much progress have students made at each grade level? Get the answers i...
Content provided by Renaissance Learning
Student Achievement Opinion Learning Recovery: The Research on Tutoring, Extended School Year, and Other Strategies
Evidence points most strongly to the value of high-dose tutoring, but other approaches have merit too, writes researcher Heather C. Hill.
5 min read
A tutor welcomes a student to a workstation
JuliarStudio/iStock/Getty Images<br/>
Student Achievement 'Learning Loss, in General, Is a Misnomer': Study Shows Kids Made Progress During COVID-19
Even though the pandemic has interrupted learning, students are still gaining ground in reading and math this year.
4 min read
Students listen to their teacher at Hawthorne Scholastic Academy in Lakeview on March 1, 2021.
Students listen to their teacher at Hawthorne Scholastic Academy in Lakeview on March 1, 2021. A new study shows that students have continued to make gains in math and reading during the pandemic.
Anthony Vazquez/Chicago Sun-Times via AP
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Whitepaper
COVID Recovery and the New Generation of Student Learners: How to Escalate Skill Growth and Prepare for a Powerful School Year Ahead
Read this white paper that includes the latest research on COVID learning loss.
Content provided by Istation