Assessment

Statistics Agency Gauging State ‘Proficiency’ Thresholds

By Sean Cavanagh — November 22, 2006 4 min read
  • Save to favorites
  • Print

The statistical arm of the U.S. Department of Education is conducting a study to see how states’ definitions of student academic proficiency compare with the way it is spelled out by the prominent national test known as “the nation’s report card.”

The goal of the study is to use that test, the National Assessment of Educational Progress, as a “yardstick” to judge the different state proficiency thresholds, said Mark S. Schneider, the commissioner of the National Center for Education Statistics, which is overseeing the project.

“For the first time, we would have a way of comparing states’ proficiency standards,” Mr. Schneider said in an interview. “Everybody’s been wanting something like this,” he added. “I’m very anxious to get this out the door.”

The proficiency levels states set on their individual assessments would be judged against those set in 4th and 8th grade reading and mathematics for NAEP, Mr. Schneider said. NCES officials hope the study is complete and ready for public release by March or April, he added.

Improving student proficiency in reading and math has emerged as a dominant national theme in K-12 education in recent years. Under the federal No Child Left Behind Act, which was signed into law in 2002, states are required to test students annually in grades 3-8, and at least once in high school, in those subjects. Scores on each state’s tests are used to evaluate whether schools and districts have made adequate yearly progress toward the goal that all students be deemed “proficient” by 2014.

But the content of state tests and the standards states use to determine which students are proficient vary enormously. In addition, the discrepancy in the percentages of students whom states report as having achieved the target is enormous, leading critics to question the reliability of states’ tests as measures of academic progress.

Partly because of those inconsistencies, NAEP results are scrutinized by educators and policymakers as a uniform benchmark against which students from every state can be judged. States are required to have a sample of their students participate in NAEP every two years in 4th and 8th grade reading and math to receive federal education funds.

Aside from proficient, NAEP’s other achievement levels for student performance are “basic” and “advanced.” NAEP standards for proficiency are widely regarded as more difficult than many of those set for state tests.

Mr. Schneider discussed the goals of the proficiency study at the quarterly meeting of the National Assessment Governing Board, on Nov. 17. The 26-member board sets NAEP policy.

Wesley D. Bruce, the assistant superintendent for assessment for the Indiana education department, said he first heard of the research project at a meeting of the Education Information Management Advisory Consortium last month. That organization, which Mr. Bruce chairs, advises the Washington-based Council of Chief State School Officers on school data issues.

Wide Scrutiny

Mr. Bruce predicted that the federal study could draw widespread attention and provide state officials with useful information. But he also said differences between the percentages of students who achieve proficiency on NAEP and those who meet that threshold on state tests could be explained partly by the different content presented at each grade level. Such explanations, however, are likely to be lost on the public when the study is released, he said.

“The concern is the rush to judgment,” Mr. Bruce said. The public might look at the results of the study and think, “If you’re not right at the NAEP level, you got it wrong,” he said.”

Mr. Schneider, who was nominated as NCES commissioner by President Bush last year, said his agency’s work on the study began before he took the job, though he has taken an interest in the project. The NCES is one of three research centers housed within the Institute of Education Sciences that Congress created in 2002 in reorganizing the Education Department’s research operations.

Mr. Schneider, who is scheduled to serve as commissioner until his term expires in 2009, said the study was likely to draw broad scrutiny. “People are going to have to look at this,” he said. They will “look at the models, look at the methods, and look at the results.”

The lead researcher is Henry I. Braun, who holds the title of distinguished presidential appointee at the Educational Testing Service, the giant nonprofit research and testing organization in Princeton, N.J. His work will build on earlier NCES-sponsored research on the alignment between NAEP and state proficiency standards, Mr. Schneider said.

The study is based partly on a comparison of the cutoff scores states use on tests to judge whether students are deemed proficient against the corresponding scores used by NAEP, Mr. Braun said. The research examines how well populations of students who are deemed proficient in individual states fared on the national assessment.

Although he believes the study is likely to spark debates among policymakers and the public, Mr. Braun also said he hoped it would advance research into the various standards used by states.

“There’s a basic uncertainty that can never be eliminated by statistical methods,” he said. The study, he added, “is the most reasonable way we can think of to [examine] state proficiency standards on a purely statistical basis.”

Related Tags:

A version of this article appeared in the November 29, 2006 edition of Education Week as Statistics Agency Gauging State ‘Proficiency’ Thresholds

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment AI May Be Coming for Standardized Testing
An international test may offer clues on how AI can help create better assessments.
4 min read
online test checklist 1610418898 brightspot
champpixs/iStock/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Design for Improvement: The Case for a New Accountability System
Assessments in more frequent intervals provide useful feedback on what students actually study. New curriculum-aligned assessments can le...
Content provided by Cognia
Assessment The 5 Burning Questions for Districts on Grading Reforms
As districts rethink grading policies, they consider the purpose of grades and how to make them more reliable measures of learning.
5 min read
Grading reform lead art
Illustration by Laura Baker/Education Week with E+ and iStock/Getty
Assessment As They Revamp Grading, Districts Try to Improve Consistency, Prevent Inflation
Districts have embraced bold changes to make grading systems more consistent, but some say they've inflated grades and sent mixed signals.
10 min read
Close crop of a teacher's hands grading a stack of papers with a red marker.
E+