NCES Finds States Lowered ‘Proficiency’ Bar

By Debra Viadero — October 29, 2009 | Corrected: February 22, 2019 6 min read

Corrected: An early version of this story incorrectly said that a majority of states had made their testing standards less rigorous between 2005 and 2007, according to the National Center for Education Statistics. An NCES report found that among states that made changes to their testing systems, tests were made less rigorous nearly half the time.

With 2014 approaching as the deadline by which states must get all their students up to “proficient” levels on state tests, a study released last week by the U.S. Department of Education’s top statistics agency suggests that some states may have lowered student-proficiency standards on such tests in recent years.

For the 47-state study, researchers under contract to the National Center for Education Statistics used student test scores to figure out where the proficiency levels on various state tests would fall on the National Assessment of Educational Progress.

Their results suggest that between 2005 and 2007, various states made their standards less rigorous in one or more grade levels or subjects in at least 26 instances. In 12 instances, particular states appeared to make their standards more stringent in one or more grade levels or subjects.

Under the federal No Child Left Behind Act, adopted nearly eight years ago, the states must show that their students have attained proficient levels of performance in reading and mathematics by the end of the 2013-14 school year. Most states are far from reaching that goal.

“I think as 2014 looms, ... clearly what a lot of states are doing is changing the bar so that a lot more students will become proficient,” said Mark S. Schneider, who served as the NCES commissioner between 2005 to 2008.

Assessment Standards in Flux

State standards for what constitutes “proficient” performance on 8th grade mathematics exams dropped in 16 states between 2005 and 2007, a federal study finds. The proficiency bar was raised over the same period in four other states.


SOURCE: National Center on Education Statistics

In 4th grade reading, the federal study found that 31 states had set proficiency scores that were lower than the cutoff for “basic” performance on NAEP. At the 8th grade level, 15 states set standards judged to be lower than the basic level on the national assessment for that grade.

But the study uses a methodology that is controversial among some testing experts. They caution that the standardized exams that states use and the more rigorous NAEP—the congressionally mandated program known as “the nation’s report card”—are too different to put on the same scale.

“If two tests don’t measure the same thing, then mapping is misleading,” said Andrew D. Ho, an assistant professor of education at the Harvard Graduate School of Education. “You can map anything onto NAEP.”

The results, nonetheless, are expected to figure in growing efforts to develop common academic standards for what K-12 students should know and be able to do. So far, 48 states are taking part in a push to craft such standards, while federal education officials are making plans to award $350 million in grants to help states plan common assessments. (“Experts, Public to Weigh In on Common Tests,” Oct. 28, 2009.)

In statement issued in response to the study, U.S. Rep. George Miller, D-Calif., the chairman of the House Education and Labor Committee, said, “It is unacceptable that many states have chosen to lower the bar rather than strive for excellence.” He said the time has come for states to adopt a common core of internationally benchmarked standards.

U.S. Secretary of Education Arne Duncan said in a statement that the NCES report “confirms what we’ve known for a long time: States are setting the bar too low.” He said states should be raising their standards, and he referred approvingly to the common-standards effort as a starting point.

Common Metric Sought

The Education Department’s statistics agency has been developing and improving its test-mapping method since 2003. According to Peggy Carr, the associate commissioner of the NCES, the aim was to develop a common metric that state policymakers could use to assess how high their states have set the bar for proficiency.

It’s a challenging task: While the national assessment is intended to measure students’ progress against what they ought to know, state tests may have different goals, such as measuring whether students are demonstrating grade-level work. NAEP also draws on nationally representative samples of students from participating states; state tests, in theory, are designed to assess learning for every child.

The new analysis draws on testing data in reading and mathematics for grades 4 and 8. It does not include Nebraska, Utah, or the District of Columbia, or results for 8th grade math in California.

Both the new report and a 2007 study that grew out of the same project show that states vary widely in what qualifies a student as proficient. A 4th grader judged to be proficient in math in Colorado or Tennessee, for example, could conceivably test at the “basic” level in Massachusetts or Missouri, where the standards were judged by the study to be most rigorous.

“This is great fodder for why we need common standards that are of high quality,” said Amber M. Winkler, the research director for the Thomas B. Fordham Institute, a Washington think tank that supports common academic standards.

In collaboration with the Northwest Evaluation Association, a testing group based in Lake Oswego, Ore., Fordham in 2007 published a report that mapped state NAEP data with state-assessment data from the Northwest group and found variations similar to those in the NCES study across states. (“Report Pans How States Set the Bar,” Oct. 10, 2007.)

“I think the intent of NCLB was to let states make that decision” on where to set the proficiency bar, added Ms. Carr, “but I don’t think anyone conceived that we would be this discrepant or this disparate in our definitions about what proficient meant.”

Changing Benchmarks

The new study is the first by the NCES to analyze whether states had changed their testing standards, either implicitly or explicitly, over time. The report stops short, however, of alleging that states may be changing their standards to meet their NCLB targets for raising students to proficiency in reading and math. Under the federal law, students are tested annually in those subjects in grades 3-8 and once during high school.

“There are different ways states could’ve gotten to this point,” Ms. Carr said. “Some states just kept their assessment exactly the way it was and just ratcheted the cut scores up or down. Others made many changes in their testing systems so, of course, they had to have new cut points on top of their new assessments.”

In all, the report says, between 12 and 18 states, depending on the grade and subject, made changes in their state testing systems between 2005 and 2007.

One was Wyoming, where the proficiency bar appeared to have slipped in both reading and math and at both grade levels studied.

“There are really no surprises in Wyoming for this report,” said William Donkersgoed, the state’s national-assesssment coordinator. “There was a change in the assessment system, and it was like comparing apples and oranges from 2005 to 2007.”

Mr. Donkersgoed said the state’s previous system, like NAEP, was developed so that schools could measure their progress against a high standard, and there were no consequences for schools that fell short. The new system, in contrast, was devised in part by panels of teachers examining what grade-level performance should look like.

Rising testing standards in Montana, on the other hand, did not seem to be linked to any dramatic, or intentional, changes in the state testing system, according to Mike Chapman, the NAEP coordinator for that state. He said state officials surmise that the change resulted from instituting tests in additional grades and offering teachers more professional development—two actions that may have had indirect ramifications for the level of that state’s 4th and 8th grade cutoff scores, which are recalibrated from year to year.

“In most states, the actual cut scores haven’t changed,” said Mr. Ho of Harvard. “What’s happening is that a change on one test is outpacing the change on another.”

The NCES researchers also looked to see whether achievement gains on state tests tracked with improvements in state students’ performance on the national assessment. For states that had not overhauled their testing systems, that was the case 60 percent to 90 percent of the time, said Ms. Carr.

“That’s good,” she added, “because that is what NAEP was intended to do.”

A version of this article appeared in the November 04, 2009 edition of Education Week as Test Rigor Drops Off, Study Finds


School & District Management Live Event EdWeek Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
How Principals Can Support Student Well-Being During COVID
Join this webinar for tips on how to support and prioritize student health and well-being during COVID.
Content provided by Unruly Studios

EdWeek Top School Jobs

President and CEO
Alexandria, Virginia
National Association of State Boards of Education
Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Arizona School Data Analyst - (AZVA)
Arizona, United States
K12 Inc.

Read Next

Assessment Biden's Testing Stance Leaves States Tough Choices. Some May Still Try to Avoid Exams
Whether to give tests in person this spring or even test students next school year instead, education leaders confront a complex path.
Flags decorate a space outside the secretary's office at the U.S. Department of Education in Washington.
Flags decorate a space outside the secretary's office at the U.S. Department of Education in Washington.
Jacquelyn Martin/AP
Assessment States Still Must Give Standardized Tests This Year, Biden Administration Announces
But the administration says it would allow states to give tests in the summer or use partial exams due to challenges related to COVID-19.
3 min read
Image of students taking a test.
Assessment Timing of Food Stamps Can Affect Students' Test Scores, Study Finds
Hungry students don't test as well, say researchers who found a link between food stamp disbursements and students' exam scores.
5 min read
A sign advertises a program that allows food stamp recipients to use their EBT cards to shop at a farmer's market in Topsham, Maine on March 17, 2017.
Food stamps can be used in some farmers' markets, as at this one in Topsham, Maine. New research shows a link between timing of the aid and student performance on key tests.
Robert F. Bukaty/AP
Assessment New Mexico Asks to Skip Student Testing Again This Year
State officials are seeking permission from federal officials to waive standardized testing for another year, citing the pandemic.
3 min read