Assessment

‘Proficient’ Mark Shifts by State, Grade, Subject

By Lynn Olson — December 03, 2003 4 min read
  • Save to favorites
  • Print

Where states set the bar for what counts as “proficient” performance on state tests differs dramatically across states and even across subjects and grades within states, a new study reveals.

The analysis by the Portland, Ore.-based Northwest Evaluation Association examines where 14 states set their proficiency levels in reading and mathematics, by comparing how students did on the state tests with how they performed on tests produced by the NWEA and taken within a month of the state exams.

Read “The State of State Standards: Research Investigating Proficiency Levels in Fourteen States,” from the Northwest Evaluation Association (requires free registration). Or, read the executive summary. (Requires Adobe’s Acrobat Reader.)

Under the federal No Child Left Behind Act, schools must meet annual targets for the percent of students who perform at or above the proficient level on state tests in reading and math in order to make “adequate yearly progress.” But the study’s findings suggest that whether schools are identified as doing a good job and students are deemed on track academically hinges as much on geography as on actual performance.

“The problem is we have a federal law with very uniform language, rules, and consequences being applied to very disparate criteria,” said Allan Olson, the president and executive director of the evaluation association.

The nonprofit organization provides computerized adaptive tests and paper-and-pencil assessments to more than 1,000 member districts in 38 states. More than 3 million students in grades 2-10 take the tests, and their performance is scored on a common measurement scale across grades.

See Also...

See the accompanying table, “Performance Standards for Math,”

By seeing where students in each state fell on the NWEA scale, and where they fell on their state tests, the test-maker was able to compare state proficiency levels against a common yardstick. In each state, at least 1,000 students in each grade and subject took part in the study.

‘Inappropriate Impressions’

While it’s not surprising that proficiency levels differ across states, subjects, and grades, said Mr. Olson, the degree to which they differ and the potential for misinterpretation are surprising. For example:

  • In 8th grade math, students who perform at the “proficient” level on state tests in Montana score at the 36th percentile on the NWEA’s exam, while students who score at the “proficient” level in Wyoming score at the 89th percentile.
  • In Arizona, 3rd graders who perform at the proficient level in math on the state test score at the 46th percentile on the NWEA exam. But 8th graders who perform at the proficient level score at the 75th percentile.

That suggests Arizona has set a much higher bar in grade 8 than in grade 3, a common occurrence across states, said Mr. Olson. As a result, a large proportion of students who are identified as on track academically in elementary school could be categorized as below proficient in grade 8, even if their learning trajectory has stayed the same. Meanwhile, states could mistakenly think that their middle schools are not doing as good a job as their elementary schools at educating children.

  • In Washington state, the proficiency level on the 4th grade reading test is at the 53rd percentile on the NWEA exam, while the proficiency level on the 4th grade math test is at the 76th percentile. That means more students will be identified as proficient in reading than in math, even if students’ performance is the same relative to that of their peers. Such results could lead the state to spend more money to help students in mathematics without needing to do so.

“You may create impressions that are inappropriate,” Mr. Olson warned. While most states do not have proficiency levels that are as divergent as the previous examples, he noted, almost all states have noticeable differences that will affect some students.

No Agreed-Upon Solution

Stuart R. Kahl, the president and chief executive officer of Measured Progress, another testing company, based in Dover, N.H., agreed that proficiency levels can vary greatly even across grades and subjects within the same state. One reason, he said, is that states often set proficiency levels, or cutoff scores, independently for each grade and subject.

But the problem is becoming more apparent as states add tests in more grades to comply with the No Child Left Behind law, Mr. Kahl said.

“What you don’t want kids doing as they go through the grades is jumping from ‘proficient’ to ‘not proficient’ just because of the way the standards are set,” he argued.

Testing experts have proposed a number of solutions to the problem, he noted. Those range from statistical techniques, such as “smoothing,” to being clearer about what students should know and be able to do in each grade in state content standards, to putting all tests on the same vertical scale, as the NWEA does.

But, Mr. Kahl cautioned, “there are strong debates over which is the best way to go.”

Regardless, he said, “states are going to have to do something.”


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
What’s Next for Teaching and Learning? Key Trends for the New School Year
The past 18 months changed the face of education forever, leaving teachers, students, and families to adapt to unprecedented challenges in teaching and learning. As we enter the third school year affected by the pandemic—and
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
iStock/Getty
Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Image of a test sheet.
sengchoy/iStock/Getty