Report Pans How States Set the Bar

By Kathleen Kennedy Manzo — October 05, 2007 5 min read
  • Save to favorites
  • Print

The random and sometimes erratic nature of state proficiency standards makes for an assessment system that is “slipshod,” obscure, and potentially unreliable, contends an outspoken think tank’s analysis of the testing benchmarks set by 26 states.

The range of expectations—3rd grade reading proficiency can mean performing at the 6th percentile on one state’s test, but at the 62nd on another’s—spells “big trouble” for standards-based improvement efforts and the No Child Left Behind Act, concludes the report, “The Proficiency Illusion,” released last week.

“A lot of states don’t expect very much of their kids, and [standards] are enormously discrepant from state to state, subject to subject, and from grade to grade,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation, which commissioned the report. “That all adds up to say the centerpiece of standards-based reform may not be up to what we expect.”

The report from Fordham reinforces growing concerns that, despite the federal mandates and the improved reporting of student-achievement data under the NCLB law, no common yardstick exists for comparing those school improvement gains across states.

“The notion that in states with less stringent standards ... students are being declared proficient and can make their way through the system … but are not ready to compete with graduates from other states that have standards that are much more stringent is a huge problem,” said Henry Braun, a researcher at Boston College who conducted a federal study earlier this year comparing state proficiency benchmarks with those of the National Assessment of Educational Progress. (“State Tests, NAEP Often a Mismatch,” June 13, 2007.)

Mr. Braun said the study from Fordham—a strong proponent of national standards and advocate for market-based reforms—is “clearly the most detailed analysis at the state level … and is consistent with, and reinforces, other reports that deal with this issue.”

But officials from some states included in the analysis said the broad statements in the report did not accurately reflect the hard work being done to raise standards and measure results, according to Gene Wilhoit, the executive director of the Council of Chief State School Officers, based in Washington.

“They felt the report took an accusatory kind of point of view that the states had been hiding standards or had not done due diligence about their program,” he said, describing a conference call state officials had with the authors of the report prior to its release. “They were concerned whether the scale was a fair standard to use.”

Tool for Comparisons

The study was conducted by the Portland, Ore.-based Northwest Evaluation Association, which administers the Measures of Academic Progress, a computerized diagnostic assessment. The MAP assessment is given to a sufficient number of students in 26 states to allow for comparisons with state tests, the report says.

Wide Variation in Cutoff Points

States set widely differing cutoff points for determining how well a student must perform in order to qualify as “proficient” in reading or math under the No Child Left Behind Act. This table lists the estimated percentile rank needed to meet that standard in 26 states, based on the Measures of Academic Progress, a computerized diagnostic test.

Click to enlarge

SOURCE: Thomas B. Fordham Foundation

NWEA researchers John Cronin, Michael Dahlin, Deborah Adkins, and Gage Kingsbury analyzed state tests and results and estimated where the cutoff scores fall on the MAP percentile scales at each grade level for math and reading.

They found that state tests for 8th graders tend to be more difficult for those students to pass than are the 3rd grade tests for the younger pupils, even after considering the differences in developmental levels and the difficulty of the content.

That inconsistency could end up giving educators and the public a false perception of how children are doing or how they should progress through the grades, according to the study.

For example, a student who reaches proficiency on the 3rd grade test in a state with low standards for that grade may not be on track to meet a more rigorous 8th grade proficiency benchmark.

The NCLB law requires states to test students annually in grades 3-8, and once in high school, in reading and math. Schools must show adequate yearly progress in moving students toward what a state defines as proficiency in those subjects or face penalties.

The report warns that students who move to a state with higher standards generally are not prepared to meet the more rigorous expectations.

Colorado, Michigan, and Wisconsin were among the states with the lowest cutoff scores across grades 3 through 8 for both subjects, while California, Massachusetts, and South Carolina tended to set the highest benchmarks.

Changing Goal Posts

Although most states have not lowered their cutoff scores to make it easier to meet NCLB requirements that they show progress toward making sure all students are “proficient” by 2014, eight states with high standards have lowered their expectations at some grade levels since the law was passed in 2001.

The study determined that the main factors in explaining states’ gains in proficiency rates might include teaching to the test, students’ motivation to do well on the tests, and adjustments to the testing standards.

Colorado sets cutoff scores equivalent to the 6th percentile on its 3rd grade reading test—a level easy enough for 94 percent of 3rd graders nationally to meet that standard—and the 13th percentile on the 8th grade test.

In math, a Colorado 3rd grader has to score at the 7th percentile, and an 8th grader at the 20th percentile, to be deemed proficient.

In South Carolina, however, 3rd graders have to score above the 60th percentile in reading and at the 71st percentile in math to reach proficiency. Eighth graders in that state must perform above the 60th percentile in reading and about the 75th percentile in math to meet the standard.

Colorado officials have been candid about setting a lower bar for the accountability purposes of NCLB than for the state accountability system. The state categorizes test-takers’ results in four groups: “advanced,” “proficient,” “partially proficient,” and “unsatisfactory.” To comply with the federal law’s AYP requirements, Colorado aims to move more students to the partially proficient category.

“We made a policy decision when NCLB came out” to use the partially proficient standard to determine AYP, said Jo O’Brien, the assistant state education commissioner for assessment. “If NCLB was really about the achievement gap, and we were putting limited federal dollars into addressing that, we wanted it to go to serving the neediest kids, those that hadn’t reached partial proficiency yet.”

The state accountability system, however, demands that schools show progress in moving more children into the proficient and advanced categories or face penalties. Ms. O’Brien noted that Colorado tends to score above the national average on the National Assessment of Educational Progress in math, reading, and science.

For South Carolina, however, the decision to set the cutoffs higher than other states has had its own drawbacks, according to state Superintendent Jim Rex.

“I think we made the right decision to have high expectations for our students,” he said. “But it has created an illusion that the gap between performance and expectations is larger in our state than other states.”


Budget & Finance Webinar Leverage New Funding Sources with Data-Informed Practices
Address the whole child using data-informed practices, gain valuable insights, and learn strategies that can benefit your district.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
ChatGPT & Education: 8 Ways AI Improves Student Outcomes
Revolutionize student success! Don't miss our expert-led webinar demonstrating practical ways AI tools will elevate learning experiences.
Content provided by Inzata
Classroom Technology K-12 Essentials Forum Tech Is Everywhere. But Is It Making Schools Better?
Join us for a lively discussion about the ways that technology is being used to improve schools and how it is falling short.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Ron DeSantis Is Running for President. What Will That Mean for K-12 Schools?
DeSantis has solidified himself as a force on school policy. His campaign will likely influence the role education plays in the election.
6 min read
Florida Gov. Ron DeSantis speaks during convocation at Liberty University, in Lynchburg, Va., on April 14, 2023.
Florida Gov. Ron DeSantis speaks during convocation at Liberty University, in Lynchburg, Va., on April 14, 2023.
Paige Dingler/The News & Advance via AP
Federal Cardona Defends Biden's Education Budget and Proposals on Student Debt and Trans Athletes
House Republicans accused Education Secretary Miguel Cardona of indoctrinating students and causing drops in test scores.
4 min read
Education Secretary Miguel Cardona speaks during a ceremony honoring the Council of Chief State School Officers' 2023 Teachers of the Year in the Rose Garden of the White House on April 24, 2023, in Washington.
Education Secretary Miguel Cardona speaks during a ceremony honoring the 2023 Teachers of the Year at the White House on April 24, 2023. He appeared before a U.S. House committee May 16, 2023, to defend the Biden administration's proposed education budget and other policies.
Andrew Harnik/AP
Federal Book Bans and Divisive Concepts Laws Will Hold U.S. Students Back, Secretary Cardona Says
Education Secretary Miguel Cardona participated in a summit this week that drew international education leaders to the nation's capital.
6 min read
Secretary of Education Miguel Cardona answers questions during an interview in his office in Washington, D.C., on Tuesday, August 23, 2022.
Secretary of Education Miguel Cardona answers questions during an interview in his office in Washington, D.C., on Tuesday, August 23, 2022.
Alyssa Schukar for Education Week
Federal Opinion The Lies America Tells Itself About Black Education
'A Nation at Risk' created a faux crisis to usher in the right's education agenda, argues Bettina L. Love.
4 min read
President Ronald Reagan is flanked by Education Secretary Terrel Bell, left, White House Policy director, during a meeting in the Cabinet Room in Washington, Feb. 23, 1984 where they discussed school discipline.
President Ronald Reagan and U.S. Secretary of Education Terrel H. Bell, left, during a meeting in the Cabinet Room, Feb. 23, 1984, where they discussed school discipline.