Accountability Opinion

Scale Score Magic! Why We Shouldn’t Rely on Passing Rates to Measure Academic Achievement

By Eduwonkette — June 24, 2008 1 min read
  • Save to favorites
  • Print

Consider this puzzle: in 2007, the average scale score on the New York State ELA Test was 661. In 2008, it is also 661. Yet the overall level of proficiency has increased by 3 percentage points, from 68% to 71%. How is this possible?

When we measure student achievement solely based on the proportion of students who have jumped over a bar, we can end up with pretty misleading picture of student performance.

Take a look at grades 3, 5, and 8 in the graph below, which shows the change in ELA average scales scores and passing rates for New York state. In each case, the average scale score increased by 2 points, or about .05 standard deviations. But the increases in the percentage of students who were proficient varied widely across those grades. In 3rd grade, there was an increase of 3 percentage points. In 5th grade, there was a much larger increase - 9.5 percentage points. And in 8th grade, though the average scale score increased, the percentage of students who were proficient actually decreased .9 percentage points.

Should we conclude that our 5th graders are much better off than they’ve been in the past, and 8th graders are falling behind? Definitely not - 5th grade just happened to hit the sweet spot of the distribution - but that’s what you’d get if you relied only on passing rates.

In short, know what you’re buying when you’re looking at passing rates. They can increase substantially by moving a small number of kids up a few points - just enough to clear the cut score. In some of the grade levels above, there are good reasons to suspect that these small moves may partially explain large jumps in proficiency on the New York State ELA test.

The opinions expressed in eduwonkette are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.