Report Pans How States Set the Bar

By Kathleen Kennedy Manzo — October 05, 2007 5 min read

The random and sometimes erratic nature of state proficiency standards makes for an assessment system that is “slipshod,” obscure, and potentially unreliable, contends an outspoken think tank’s analysis of the testing benchmarks set by 26 states.

The range of expectations—3rd grade reading proficiency can mean performing at the 6th percentile on one state’s test, but at the 62nd on another’s—spells “big trouble” for standards-based improvement efforts and the No Child Left Behind Act, concludes the report, “The Proficiency Illusion,” released last week.

“A lot of states don’t expect very much of their kids, and [standards] are enormously discrepant from state to state, subject to subject, and from grade to grade,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation, which commissioned the report. “That all adds up to say the centerpiece of standards-based reform may not be up to what we expect.”

The report from Fordham reinforces growing concerns that, despite the federal mandates and the improved reporting of student-achievement data under the NCLB law, no common yardstick exists for comparing those school improvement gains across states.

“The notion that in states with less stringent standards ... students are being declared proficient and can make their way through the system … but are not ready to compete with graduates from other states that have standards that are much more stringent is a huge problem,” said Henry Braun, a researcher at Boston College who conducted a federal study earlier this year comparing state proficiency benchmarks with those of the National Assessment of Educational Progress. (“State Tests, NAEP Often a Mismatch,” June 13, 2007.)

Mr. Braun said the study from Fordham—a strong proponent of national standards and advocate for market-based reforms—is “clearly the most detailed analysis at the state level … and is consistent with, and reinforces, other reports that deal with this issue.”

But officials from some states included in the analysis said the broad statements in the report did not accurately reflect the hard work being done to raise standards and measure results, according to Gene Wilhoit, the executive director of the Council of Chief State School Officers, based in Washington.

“They felt the report took an accusatory kind of point of view that the states had been hiding standards or had not done due diligence about their program,” he said, describing a conference call state officials had with the authors of the report prior to its release. “They were concerned whether the scale was a fair standard to use.”

Tool for Comparisons

The study was conducted by the Portland, Ore.-based Northwest Evaluation Association, which administers the Measures of Academic Progress, a computerized diagnostic assessment. The MAP assessment is given to a sufficient number of students in 26 states to allow for comparisons with state tests, the report says.

Wide Variation in Cutoff Points

States set widely differing cutoff points for determining how well a student must perform in order to qualify as “proficient” in reading or math under the No Child Left Behind Act. This table lists the estimated percentile rank needed to meet that standard in 26 states, based on the Measures of Academic Progress, a computerized diagnostic test.

Click to enlarge

SOURCE: Thomas B. Fordham Foundation

NWEA researchers John Cronin, Michael Dahlin, Deborah Adkins, and Gage Kingsbury analyzed state tests and results and estimated where the cutoff scores fall on the MAP percentile scales at each grade level for math and reading.

They found that state tests for 8th graders tend to be more difficult for those students to pass than are the 3rd grade tests for the younger pupils, even after considering the differences in developmental levels and the difficulty of the content.

That inconsistency could end up giving educators and the public a false perception of how children are doing or how they should progress through the grades, according to the study.

For example, a student who reaches proficiency on the 3rd grade test in a state with low standards for that grade may not be on track to meet a more rigorous 8th grade proficiency benchmark.

The NCLB law requires states to test students annually in grades 3-8, and once in high school, in reading and math. Schools must show adequate yearly progress in moving students toward what a state defines as proficiency in those subjects or face penalties.

The report warns that students who move to a state with higher standards generally are not prepared to meet the more rigorous expectations.

Colorado, Michigan, and Wisconsin were among the states with the lowest cutoff scores across grades 3 through 8 for both subjects, while California, Massachusetts, and South Carolina tended to set the highest benchmarks.

Changing Goal Posts

Although most states have not lowered their cutoff scores to make it easier to meet NCLB requirements that they show progress toward making sure all students are “proficient” by 2014, eight states with high standards have lowered their expectations at some grade levels since the law was passed in 2001.

The study determined that the main factors in explaining states’ gains in proficiency rates might include teaching to the test, students’ motivation to do well on the tests, and adjustments to the testing standards.

Colorado sets cutoff scores equivalent to the 6th percentile on its 3rd grade reading test—a level easy enough for 94 percent of 3rd graders nationally to meet that standard—and the 13th percentile on the 8th grade test.

In math, a Colorado 3rd grader has to score at the 7th percentile, and an 8th grader at the 20th percentile, to be deemed proficient.

In South Carolina, however, 3rd graders have to score above the 60th percentile in reading and at the 71st percentile in math to reach proficiency. Eighth graders in that state must perform above the 60th percentile in reading and about the 75th percentile in math to meet the standard.

Colorado officials have been candid about setting a lower bar for the accountability purposes of NCLB than for the state accountability system. The state categorizes test-takers’ results in four groups: “advanced,” “proficient,” “partially proficient,” and “unsatisfactory.” To comply with the federal law’s AYP requirements, Colorado aims to move more students to the partially proficient category.

“We made a policy decision when NCLB came out” to use the partially proficient standard to determine AYP, said Jo O’Brien, the assistant state education commissioner for assessment. “If NCLB was really about the achievement gap, and we were putting limited federal dollars into addressing that, we wanted it to go to serving the neediest kids, those that hadn’t reached partial proficiency yet.”

The state accountability system, however, demands that schools show progress in moving more children into the proficient and advanced categories or face penalties. Ms. O’Brien noted that Colorado tends to score above the national average on the National Assessment of Educational Progress in math, reading, and science.

For South Carolina, however, the decision to set the cutoffs higher than other states has had its own drawbacks, according to state Superintendent Jim Rex.

“I think we made the right decision to have high expectations for our students,” he said. “But it has created an illusion that the gap between performance and expectations is larger in our state than other states.”

Let us know what you think!

We’re looking for feedback on our new site to make sure we continue to provide you the best experience.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Future of Work Webinar
Digital Literacy Strategies to Promote Equity
Our new world has only increased our students’ dependence on technology. This makes digital literacy no longer a “nice to have” but a “need to have.” How do we ensure that every student can navigate
Content provided by Learning.com
Mathematics Online Summit Teaching Math in a Pandemic
Attend this online summit to ask questions about how COVID-19 has affected achievement, instruction, assessment, and engagement in math.
School & District Management Webinar Examining the Evidence: Catching Kids Up at a Distance
As districts, schools, and families navigate a new normal following the abrupt end of in-person schooling this spring, students’ learning opportunities vary enormously across the nation. Access to devices and broadband internet and a secure

EdWeek Top School Jobs

Great Oaks AmeriCorps Fellow August 2021 - June 2022
New York City, New York (US)
Great Oaks Charter Schools
Great Oaks AmeriCorps Fellow August 2021 - June 2022
New York City, New York (US)
Great Oaks Charter Schools
Data Analyst
New York, NY, US
New Visions for Public Schools

Read Next

Federal Biden Calls for $130 Billion in New K-12 Relief, Scaled Up Testing, Vaccination Efforts
President-elect Joe Biden proposed new aid for schools as part of a broader COVID-19 relief plan, which will require congressional approval.
5 min read
First-grade teacher Megan Garner-Jones, left, and Principal Cynthia Eisner silent clap for their students participating remotely and in-person at School 16, in Yonkers, N.Y., on Oct. 20, 2020.
First-grade teacher Megan Garner-Jones, left, and Principal Cynthia Eisner silent clap for their students participating remotely and in-person at School 16, in Yonkers, N.Y.
Mary Altaffer/AP
Federal Who Is Miguel Cardona? Education Secretary Pick Has Roots in Classroom, Principal's Office
Many who've worked with Joe Biden's pick for education secretary say he's ready for what would be a giant step up.
15 min read
Miguel Cardona, first-time teacher, in his fourth-grade classroom at Israel Putnam School in Meriden, Ct. in August of 1998.
Miguel Cardona, chosen to lead the U.S. Department of Education, photographed in his 4th-grade classroom at Israel Putnam School in Meriden, Conn., in 1998.
Courtesy of the Record-Journal
Federal Obama Education Staff Involved in Race to the Top, Civil Rights Join Biden's White House
Both Catherine Lhamon and Carmel Martin will serve on President-elect Joe Biden's Domestic Policy Council.
4 min read
Federal Opinion What Conservatives Should Be for When It Comes to Education
Education is ultimately about opportunity, community, and empowerment, and nothing should resonate more deeply with the conservative heart.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty