Assessment

‘Proficient’ Mark Shifts by State, Grade, Subject

By Lynn Olson — December 03, 2003 4 min read

Where states set the bar for what counts as “proficient” performance on state tests differs dramatically across states and even across subjects and grades within states, a new study reveals.

The analysis by the Portland, Ore.-based Northwest Evaluation Association examines where 14 states set their proficiency levels in reading and mathematics, by comparing how students did on the state tests with how they performed on tests produced by the NWEA and taken within a month of the state exams.

Read “The State of State Standards: Research Investigating Proficiency Levels in Fourteen States,” from the Northwest Evaluation Association (requires free registration). Or, read the executive summary. (Requires Adobe’s Acrobat Reader.)

Under the federal No Child Left Behind Act, schools must meet annual targets for the percent of students who perform at or above the proficient level on state tests in reading and math in order to make “adequate yearly progress.” But the study’s findings suggest that whether schools are identified as doing a good job and students are deemed on track academically hinges as much on geography as on actual performance.

“The problem is we have a federal law with very uniform language, rules, and consequences being applied to very disparate criteria,” said Allan Olson, the president and executive director of the evaluation association.

The nonprofit organization provides computerized adaptive tests and paper-and-pencil assessments to more than 1,000 member districts in 38 states. More than 3 million students in grades 2-10 take the tests, and their performance is scored on a common measurement scale across grades.

See Also...

See the accompanying table, “Performance Standards for Math,”

By seeing where students in each state fell on the NWEA scale, and where they fell on their state tests, the test-maker was able to compare state proficiency levels against a common yardstick. In each state, at least 1,000 students in each grade and subject took part in the study.

‘Inappropriate Impressions’

While it’s not surprising that proficiency levels differ across states, subjects, and grades, said Mr. Olson, the degree to which they differ and the potential for misinterpretation are surprising. For example:

  • In 8th grade math, students who perform at the “proficient” level on state tests in Montana score at the 36th percentile on the NWEA’s exam, while students who score at the “proficient” level in Wyoming score at the 89th percentile.
  • In Arizona, 3rd graders who perform at the proficient level in math on the state test score at the 46th percentile on the NWEA exam. But 8th graders who perform at the proficient level score at the 75th percentile.

That suggests Arizona has set a much higher bar in grade 8 than in grade 3, a common occurrence across states, said Mr. Olson. As a result, a large proportion of students who are identified as on track academically in elementary school could be categorized as below proficient in grade 8, even if their learning trajectory has stayed the same. Meanwhile, states could mistakenly think that their middle schools are not doing as good a job as their elementary schools at educating children.

  • In Washington state, the proficiency level on the 4th grade reading test is at the 53rd percentile on the NWEA exam, while the proficiency level on the 4th grade math test is at the 76th percentile. That means more students will be identified as proficient in reading than in math, even if students’ performance is the same relative to that of their peers. Such results could lead the state to spend more money to help students in mathematics without needing to do so.

“You may create impressions that are inappropriate,” Mr. Olson warned. While most states do not have proficiency levels that are as divergent as the previous examples, he noted, almost all states have noticeable differences that will affect some students.

No Agreed-Upon Solution

Stuart R. Kahl, the president and chief executive officer of Measured Progress, another testing company, based in Dover, N.H., agreed that proficiency levels can vary greatly even across grades and subjects within the same state. One reason, he said, is that states often set proficiency levels, or cutoff scores, independently for each grade and subject.

But the problem is becoming more apparent as states add tests in more grades to comply with the No Child Left Behind law, Mr. Kahl said.

“What you don’t want kids doing as they go through the grades is jumping from ‘proficient’ to ‘not proficient’ just because of the way the standards are set,” he argued.

Testing experts have proposed a number of solutions to the problem, he noted. Those range from statistical techniques, such as “smoothing,” to being clearer about what students should know and be able to do in each grade in state content standards, to putting all tests on the same vertical scale, as the NWEA does.

But, Mr. Kahl cautioned, “there are strong debates over which is the best way to go.”

Regardless, he said, “states are going to have to do something.”

Events

Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How to Power Your Curriculum With Digital Books
Register for this can’t miss session looking at best practices for utilizing digital books to support their curriculum.
Content provided by OverDrive
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Embracing Student Engagement: The Pathway to Post-Pandemic Learning
As schools emerge from remote learning, educators are understandably worried about content and skills that students would otherwise have learned under normal circumstances. This raises the very real possibility that children will face endless hours
Content provided by Newsela

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion Grading Has Always Been an Imperfect Exercise. COVID-19 Made It Worse
It’s hard reducing the complexity of each student’s social, emotional, and academic learning to a letter grade. Maybe we’re doing it wrong.
Lory Walker Peroff
4 min read
A student's grades are unknown
Robert Neubecker for Education Week
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Facing the Future Together: Digital Innovative Solutions
Join us to discuss how digital innovative solutions can enrich the educational experience in the K-12 environment. We’ll share how these ...
Content provided by Pearson
Assessment Opinion What Federally Mandated State Tests Are Good For (And What They Aren’t)
Spring 2021 testing is happening. That can be a good thing—if the goal is about more than school accountability.
Stuart Kahl
5 min read
Two people analyze test data
Visual Generation/iStock/Getty
Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty