Assessment

NCES Finds States Lowered ‘Proficiency’ Bar

Results May Help Drive Common-Standards Effort
By Debra Viadero — October 29, 2009 | Corrected: February 22, 2019 6 min read
  • Save to favorites
  • Print

Corrected: An early version of this story incorrectly said that a majority of states had made their testing standards less rigorous between 2005 and 2007, according to the National Center for Education Statistics. An NCES report found that among states that made changes to their testing systems, tests were made less rigorous nearly half the time.

With 2014 approaching as the deadline by which states must get all their students up to “proficient” levels on state tests, a study released last week by the U.S. Department of Education’s top statistics agency suggests that some states may have lowered student-proficiency standards on such tests in recent years.

For the 47-state study, researchers under contract to the National Center for Education Statistics used student test scores to figure out where the proficiency levels on various state tests would fall on the National Assessment of Educational Progress.

Their results suggest that between 2005 and 2007, various states made their standards less rigorous in one or more grade levels or subjects in at least 26 instances. In 12 instances, particular states appeared to make their standards more stringent in one or more grade levels or subjects.

Under the federal No Child Left Behind Act, adopted nearly eight years ago, the states must show that their students have attained proficient levels of performance in reading and mathematics by the end of the 2013-14 school year. Most states are far from reaching that goal.

“I think as 2014 looms, ... clearly what a lot of states are doing is changing the bar so that a lot more students will become proficient,” said Mark S. Schneider, who served as the NCES commissioner between 2005 to 2008.

Assessment Standards in Flux

State standards for what constitutes “proficient” performance on 8th grade mathematics exams dropped in 16 states between 2005 and 2007, a federal study finds. The proficiency bar was raised over the same period in four other states.

BRIC ARCHIVE

SOURCE: National Center on Education Statistics

In 4th grade reading, the federal study found that 31 states had set proficiency scores that were lower than the cutoff for “basic” performance on NAEP. At the 8th grade level, 15 states set standards judged to be lower than the basic level on the national assessment for that grade.

But the study uses a methodology that is controversial among some testing experts. They caution that the standardized exams that states use and the more rigorous NAEP—the congressionally mandated program known as “the nation’s report card”—are too different to put on the same scale.

“If two tests don’t measure the same thing, then mapping is misleading,” said Andrew D. Ho, an assistant professor of education at the Harvard Graduate School of Education. “You can map anything onto NAEP.”

The results, nonetheless, are expected to figure in growing efforts to develop common academic standards for what K-12 students should know and be able to do. So far, 48 states are taking part in a push to craft such standards, while federal education officials are making plans to award $350 million in grants to help states plan common assessments. (“Experts, Public to Weigh In on Common Tests,” Oct. 28, 2009.)

In statement issued in response to the study, U.S. Rep. George Miller, D-Calif., the chairman of the House Education and Labor Committee, said, “It is unacceptable that many states have chosen to lower the bar rather than strive for excellence.” He said the time has come for states to adopt a common core of internationally benchmarked standards.

U.S. Secretary of Education Arne Duncan said in a statement that the NCES report “confirms what we’ve known for a long time: States are setting the bar too low.” He said states should be raising their standards, and he referred approvingly to the common-standards effort as a starting point.

Common Metric Sought

The Education Department’s statistics agency has been developing and improving its test-mapping method since 2003. According to Peggy Carr, the associate commissioner of the NCES, the aim was to develop a common metric that state policymakers could use to assess how high their states have set the bar for proficiency.

It’s a challenging task: While the national assessment is intended to measure students’ progress against what they ought to know, state tests may have different goals, such as measuring whether students are demonstrating grade-level work. NAEP also draws on nationally representative samples of students from participating states; state tests, in theory, are designed to assess learning for every child.

The new analysis draws on testing data in reading and mathematics for grades 4 and 8. It does not include Nebraska, Utah, or the District of Columbia, or results for 8th grade math in California.

Both the new report and a 2007 study that grew out of the same project show that states vary widely in what qualifies a student as proficient. A 4th grader judged to be proficient in math in Colorado or Tennessee, for example, could conceivably test at the “basic” level in Massachusetts or Missouri, where the standards were judged by the study to be most rigorous.

“This is great fodder for why we need common standards that are of high quality,” said Amber M. Winkler, the research director for the Thomas B. Fordham Institute, a Washington think tank that supports common academic standards.

In collaboration with the Northwest Evaluation Association, a testing group based in Lake Oswego, Ore., Fordham in 2007 published a report that mapped state NAEP data with state-assessment data from the Northwest group and found variations similar to those in the NCES study across states. (“Report Pans How States Set the Bar,” Oct. 10, 2007.)

“I think the intent of NCLB was to let states make that decision” on where to set the proficiency bar, added Ms. Carr, “but I don’t think anyone conceived that we would be this discrepant or this disparate in our definitions about what proficient meant.”

Changing Benchmarks

The new study is the first by the NCES to analyze whether states had changed their testing standards, either implicitly or explicitly, over time. The report stops short, however, of alleging that states may be changing their standards to meet their NCLB targets for raising students to proficiency in reading and math. Under the federal law, students are tested annually in those subjects in grades 3-8 and once during high school.

“There are different ways states could’ve gotten to this point,” Ms. Carr said. “Some states just kept their assessment exactly the way it was and just ratcheted the cut scores up or down. Others made many changes in their testing systems so, of course, they had to have new cut points on top of their new assessments.”

In all, the report says, between 12 and 18 states, depending on the grade and subject, made changes in their state testing systems between 2005 and 2007.

One was Wyoming, where the proficiency bar appeared to have slipped in both reading and math and at both grade levels studied.

“There are really no surprises in Wyoming for this report,” said William Donkersgoed, the state’s national-assesssment coordinator. “There was a change in the assessment system, and it was like comparing apples and oranges from 2005 to 2007.”

Mr. Donkersgoed said the state’s previous system, like NAEP, was developed so that schools could measure their progress against a high standard, and there were no consequences for schools that fell short. The new system, in contrast, was devised in part by panels of teachers examining what grade-level performance should look like.

Rising testing standards in Montana, on the other hand, did not seem to be linked to any dramatic, or intentional, changes in the state testing system, according to Mike Chapman, the NAEP coordinator for that state. He said state officials surmise that the change resulted from instituting tests in additional grades and offering teachers more professional development—two actions that may have had indirect ramifications for the level of that state’s 4th and 8th grade cutoff scores, which are recalibrated from year to year.

“In most states, the actual cut scores haven’t changed,” said Mr. Ho of Harvard. “What’s happening is that a change on one test is outpacing the change on another.”

The NCES researchers also looked to see whether achievement gains on state tests tracked with improvements in state students’ performance on the national assessment. For states that had not overhauled their testing systems, that was the case 60 percent to 90 percent of the time, said Ms. Carr.

“That’s good,” she added, “because that is what NAEP was intended to do.”

A version of this article appeared in the November 04, 2009 edition of Education Week as Test Rigor Drops Off, Study Finds

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Design for Improvement: The Case for a New Accountability System
Assessments in more frequent intervals provide useful feedback on what students actually study. New curriculum-aligned assessments can le...
Content provided by Cognia
Assessment The 5 Burning Questions for Districts on Grading Reforms
As districts rethink grading policies, they consider the purpose of grades and how to make them more reliable measures of learning.
5 min read
Grading reform lead art
Illustration by Laura Baker/Education Week with E+ and iStock/Getty
Assessment As They Revamp Grading, Districts Try to Improve Consistency, Prevent Inflation
Districts have embraced bold changes to make grading systems more consistent, but some say they've inflated grades and sent mixed signals.
10 min read
Close crop of a teacher's hands grading a stack of papers with a red marker.
E+
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Sponsor
Fewer, Better Assessments: Rethinking Assessments and Reducing Data Fatigue
Imagine a classroom where data isn't just a report card, but a map leading students to their full potential. That's the kind of learning experience we envision at ANet, alongside educators
Content provided by Achievement Network
Superintendent Dr. Kelly Aramaki - Watch how ANet helps educators
Photo provided by Achievement Network