Assessment

‘Proficient’ Mark Shifts by State, Grade, Subject

By Lynn Olson — December 03, 2003 4 min read
  • Save to favorites
  • Print

Where states set the bar for what counts as “proficient” performance on state tests differs dramatically across states and even across subjects and grades within states, a new study reveals.

The analysis by the Portland, Ore.-based Northwest Evaluation Association examines where 14 states set their proficiency levels in reading and mathematics, by comparing how students did on the state tests with how they performed on tests produced by the NWEA and taken within a month of the state exams.

Read “The State of State Standards: Research Investigating Proficiency Levels in Fourteen States,” from the Northwest Evaluation Association (requires free registration). Or, read the executive summary. (Requires Adobe’s Acrobat Reader.)

Under the federal No Child Left Behind Act, schools must meet annual targets for the percent of students who perform at or above the proficient level on state tests in reading and math in order to make “adequate yearly progress.” But the study’s findings suggest that whether schools are identified as doing a good job and students are deemed on track academically hinges as much on geography as on actual performance.

“The problem is we have a federal law with very uniform language, rules, and consequences being applied to very disparate criteria,” said Allan Olson, the president and executive director of the evaluation association.

The nonprofit organization provides computerized adaptive tests and paper-and-pencil assessments to more than 1,000 member districts in 38 states. More than 3 million students in grades 2-10 take the tests, and their performance is scored on a common measurement scale across grades.

See Also...

See the accompanying table, “Performance Standards for Math,”

By seeing where students in each state fell on the NWEA scale, and where they fell on their state tests, the test-maker was able to compare state proficiency levels against a common yardstick. In each state, at least 1,000 students in each grade and subject took part in the study.

‘Inappropriate Impressions’

While it’s not surprising that proficiency levels differ across states, subjects, and grades, said Mr. Olson, the degree to which they differ and the potential for misinterpretation are surprising. For example:

  • In 8th grade math, students who perform at the “proficient” level on state tests in Montana score at the 36th percentile on the NWEA’s exam, while students who score at the “proficient” level in Wyoming score at the 89th percentile.
  • In Arizona, 3rd graders who perform at the proficient level in math on the state test score at the 46th percentile on the NWEA exam. But 8th graders who perform at the proficient level score at the 75th percentile.

That suggests Arizona has set a much higher bar in grade 8 than in grade 3, a common occurrence across states, said Mr. Olson. As a result, a large proportion of students who are identified as on track academically in elementary school could be categorized as below proficient in grade 8, even if their learning trajectory has stayed the same. Meanwhile, states could mistakenly think that their middle schools are not doing as good a job as their elementary schools at educating children.

  • In Washington state, the proficiency level on the 4th grade reading test is at the 53rd percentile on the NWEA exam, while the proficiency level on the 4th grade math test is at the 76th percentile. That means more students will be identified as proficient in reading than in math, even if students’ performance is the same relative to that of their peers. Such results could lead the state to spend more money to help students in mathematics without needing to do so.

“You may create impressions that are inappropriate,” Mr. Olson warned. While most states do not have proficiency levels that are as divergent as the previous examples, he noted, almost all states have noticeable differences that will affect some students.

No Agreed-Upon Solution

Stuart R. Kahl, the president and chief executive officer of Measured Progress, another testing company, based in Dover, N.H., agreed that proficiency levels can vary greatly even across grades and subjects within the same state. One reason, he said, is that states often set proficiency levels, or cutoff scores, independently for each grade and subject.

But the problem is becoming more apparent as states add tests in more grades to comply with the No Child Left Behind law, Mr. Kahl said.

“What you don’t want kids doing as they go through the grades is jumping from ‘proficient’ to ‘not proficient’ just because of the way the standards are set,” he argued.

Testing experts have proposed a number of solutions to the problem, he noted. Those range from statistical techniques, such as “smoothing,” to being clearer about what students should know and be able to do in each grade in state content standards, to putting all tests on the same vertical scale, as the NWEA does.

But, Mr. Kahl cautioned, “there are strong debates over which is the best way to go.”

Regardless, he said, “states are going to have to do something.”

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Strategies for Improving Student Outcomes with Teacher-Student Relationships
Explore strategies for strengthening teacher-student relationships and hear how districts are putting these methods into practice to support positive student outcomes.
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Classroom Technology Webinar
Transform Teaching and Learning with AI
Increase productivity and support innovative teaching with AI in the classroom.
Content provided by Promethean
Curriculum Webinar Computer Science Education Movement Gathers Momentum. How Should Schools React?
Discover how schools can expand opportunities for students to study computer science education.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Latest Round of Federal Grants Aims to Make States' Assessments More Equitable, Precise
The U.S. Department of Education awarded over $29 million in competitive grants to 10 state education agencies.
2 min read
Assessment review data 599911460
vladwei/iStock/Getty<br/>
Assessment Opinion Are There Better Ways Than Standardized Tests to Assess Students? Educators Think So
Student portfolios and school community surveys are but two of the many alternatives to standardized tests.
3 min read
Illustration of students in virus environment facing wave of test sheets.
Collage by Vanessa Solis/Education Week (Images: iStock/DigitalVision Vectors/Getty)
Assessment Letter to the Editor We Need NAEP
The president and CEO of Knowledge Alliance responds to a recent opinion essay's criticism of the National Assessment of Educational Progress.
1 min read
Illustration of an open laptop receiving an email.
iStock/Getty
Assessment Letter to the Editor 2022 Assessment ‘Most Important’ Ever
The executive director of the National Assessment Governing Board responds to criticism of NAEP in this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.
iStock/Getty