Assessment

‘Proficient’ Mark Shifts by State, Grade, Subject

By Lynn Olson — December 03, 2003 4 min read
  • Save to favorites
  • Print

Where states set the bar for what counts as “proficient” performance on state tests differs dramatically across states and even across subjects and grades within states, a new study reveals.

The analysis by the Portland, Ore.-based Northwest Evaluation Association examines where 14 states set their proficiency levels in reading and mathematics, by comparing how students did on the state tests with how they performed on tests produced by the NWEA and taken within a month of the state exams.

Read “The State of State Standards: Research Investigating Proficiency Levels in Fourteen States,” from the Northwest Evaluation Association (requires free registration). Or, read the executive summary. (Requires Adobe’s Acrobat Reader.)

Under the federal No Child Left Behind Act, schools must meet annual targets for the percent of students who perform at or above the proficient level on state tests in reading and math in order to make “adequate yearly progress.” But the study’s findings suggest that whether schools are identified as doing a good job and students are deemed on track academically hinges as much on geography as on actual performance.

“The problem is we have a federal law with very uniform language, rules, and consequences being applied to very disparate criteria,” said Allan Olson, the president and executive director of the evaluation association.

The nonprofit organization provides computerized adaptive tests and paper-and-pencil assessments to more than 1,000 member districts in 38 states. More than 3 million students in grades 2-10 take the tests, and their performance is scored on a common measurement scale across grades.

See Also...

See the accompanying table, “Performance Standards for Math,”

By seeing where students in each state fell on the NWEA scale, and where they fell on their state tests, the test-maker was able to compare state proficiency levels against a common yardstick. In each state, at least 1,000 students in each grade and subject took part in the study.

‘Inappropriate Impressions’

While it’s not surprising that proficiency levels differ across states, subjects, and grades, said Mr. Olson, the degree to which they differ and the potential for misinterpretation are surprising. For example:

  • In 8th grade math, students who perform at the “proficient” level on state tests in Montana score at the 36th percentile on the NWEA’s exam, while students who score at the “proficient” level in Wyoming score at the 89th percentile.
  • In Arizona, 3rd graders who perform at the proficient level in math on the state test score at the 46th percentile on the NWEA exam. But 8th graders who perform at the proficient level score at the 75th percentile.

That suggests Arizona has set a much higher bar in grade 8 than in grade 3, a common occurrence across states, said Mr. Olson. As a result, a large proportion of students who are identified as on track academically in elementary school could be categorized as below proficient in grade 8, even if their learning trajectory has stayed the same. Meanwhile, states could mistakenly think that their middle schools are not doing as good a job as their elementary schools at educating children.

  • In Washington state, the proficiency level on the 4th grade reading test is at the 53rd percentile on the NWEA exam, while the proficiency level on the 4th grade math test is at the 76th percentile. That means more students will be identified as proficient in reading than in math, even if students’ performance is the same relative to that of their peers. Such results could lead the state to spend more money to help students in mathematics without needing to do so.

“You may create impressions that are inappropriate,” Mr. Olson warned. While most states do not have proficiency levels that are as divergent as the previous examples, he noted, almost all states have noticeable differences that will affect some students.

No Agreed-Upon Solution

Stuart R. Kahl, the president and chief executive officer of Measured Progress, another testing company, based in Dover, N.H., agreed that proficiency levels can vary greatly even across grades and subjects within the same state. One reason, he said, is that states often set proficiency levels, or cutoff scores, independently for each grade and subject.

But the problem is becoming more apparent as states add tests in more grades to comply with the No Child Left Behind law, Mr. Kahl said.

“What you don’t want kids doing as they go through the grades is jumping from ‘proficient’ to ‘not proficient’ just because of the way the standards are set,” he argued.

Testing experts have proposed a number of solutions to the problem, he noted. Those range from statistical techniques, such as “smoothing,” to being clearer about what students should know and be able to do in each grade in state content standards, to putting all tests on the same vertical scale, as the NWEA does.

But, Mr. Kahl cautioned, “there are strong debates over which is the best way to go.”

Regardless, he said, “states are going to have to do something.”

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Budget & Finance Webinar
The ABCs of ESSER: How to Make the Most of Relief Funds Before They Expire
Join a diverse group of K-12 experts to learn how to leverage federal funds before they expire and improve student learning environments.
Content provided by Johnson Controls
Science K-12 Essentials Forum How To Teach STEM Problem Solving Skills to All K-12 Students
Join experts for a look at how experts are integrating the teaching of problem solving and entrepreneurial thinking into STEM instruction.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Modernizing Principal Support: The Road to More Connected and Effective Leaders
When principals are better equipped to lead, support, and maintain high levels of teaching and learning, outcomes for students are improved.
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment The 'Nation's Report Card' Is Getting an Overhaul: 5 Things to Know
The leaders of NAEP have big plans for making the test more nimble, flexible, and useful.
9 min read
Image of a bank of computers in a library.
baona/E+
Assessment Opinion What the Digital SAT Will Mean for Students and Educators
The college-admissions test will be fully digital by 2024. Priscilla Rodriguez from the College Board discusses the change.
6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Assessment Opinion Searching for Common Ground: What Makes a Good Test?
Rick Hess and USC Dean Pedro Noguera discuss standardized testing—what it’s for, where it’s gone wrong, and how to improve it.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Assessment Spotlight Spotlight on Assessment in 2022
This Spotlight will help you understand how to use assessment data to guide student learning and examine the debate over standardized tests.