Federal

Report Pans How States Set the Bar

By Kathleen Kennedy Manzo — October 05, 2007 5 min read
  • Save to favorites
  • Print

The random and sometimes erratic nature of state proficiency standards makes for an assessment system that is “slipshod,” obscure, and potentially unreliable, contends an outspoken think tank’s analysis of the testing benchmarks set by 26 states.

The range of expectations—3rd grade reading proficiency can mean performing at the 6th percentile on one state’s test, but at the 62nd on another’s—spells “big trouble” for standards-based improvement efforts and the No Child Left Behind Act, concludes the report, “The Proficiency Illusion,” released last week.

“A lot of states don’t expect very much of their kids, and [standards] are enormously discrepant from state to state, subject to subject, and from grade to grade,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation, which commissioned the report. “That all adds up to say the centerpiece of standards-based reform may not be up to what we expect.”

The report from Fordham reinforces growing concerns that, despite the federal mandates and the improved reporting of student-achievement data under the NCLB law, no common yardstick exists for comparing those school improvement gains across states.

“The notion that in states with less stringent standards ... students are being declared proficient and can make their way through the system … but are not ready to compete with graduates from other states that have standards that are much more stringent is a huge problem,” said Henry Braun, a researcher at Boston College who conducted a federal study earlier this year comparing state proficiency benchmarks with those of the National Assessment of Educational Progress. (“State Tests, NAEP Often a Mismatch,” June 13, 2007.)

Mr. Braun said the study from Fordham—a strong proponent of national standards and advocate for market-based reforms—is “clearly the most detailed analysis at the state level … and is consistent with, and reinforces, other reports that deal with this issue.”

But officials from some states included in the analysis said the broad statements in the report did not accurately reflect the hard work being done to raise standards and measure results, according to Gene Wilhoit, the executive director of the Council of Chief State School Officers, based in Washington.

“They felt the report took an accusatory kind of point of view that the states had been hiding standards or had not done due diligence about their program,” he said, describing a conference call state officials had with the authors of the report prior to its release. “They were concerned whether the scale was a fair standard to use.”

Tool for Comparisons

The study was conducted by the Portland, Ore.-based Northwest Evaluation Association, which administers the Measures of Academic Progress, a computerized diagnostic assessment. The MAP assessment is given to a sufficient number of students in 26 states to allow for comparisons with state tests, the report says.

Wide Variation in Cutoff Points

States set widely differing cutoff points for determining how well a student must perform in order to qualify as “proficient” in reading or math under the No Child Left Behind Act. This table lists the estimated percentile rank needed to meet that standard in 26 states, based on the Measures of Academic Progress, a computerized diagnostic test.

Click to enlarge

SOURCE: Thomas B. Fordham Foundation

NWEA researchers John Cronin, Michael Dahlin, Deborah Adkins, and Gage Kingsbury analyzed state tests and results and estimated where the cutoff scores fall on the MAP percentile scales at each grade level for math and reading.

They found that state tests for 8th graders tend to be more difficult for those students to pass than are the 3rd grade tests for the younger pupils, even after considering the differences in developmental levels and the difficulty of the content.

That inconsistency could end up giving educators and the public a false perception of how children are doing or how they should progress through the grades, according to the study.

For example, a student who reaches proficiency on the 3rd grade test in a state with low standards for that grade may not be on track to meet a more rigorous 8th grade proficiency benchmark.

The NCLB law requires states to test students annually in grades 3-8, and once in high school, in reading and math. Schools must show adequate yearly progress in moving students toward what a state defines as proficiency in those subjects or face penalties.

The report warns that students who move to a state with higher standards generally are not prepared to meet the more rigorous expectations.

Colorado, Michigan, and Wisconsin were among the states with the lowest cutoff scores across grades 3 through 8 for both subjects, while California, Massachusetts, and South Carolina tended to set the highest benchmarks.

Changing Goal Posts

Although most states have not lowered their cutoff scores to make it easier to meet NCLB requirements that they show progress toward making sure all students are “proficient” by 2014, eight states with high standards have lowered their expectations at some grade levels since the law was passed in 2001.

The study determined that the main factors in explaining states’ gains in proficiency rates might include teaching to the test, students’ motivation to do well on the tests, and adjustments to the testing standards.

Colorado sets cutoff scores equivalent to the 6th percentile on its 3rd grade reading test—a level easy enough for 94 percent of 3rd graders nationally to meet that standard—and the 13th percentile on the 8th grade test.

In math, a Colorado 3rd grader has to score at the 7th percentile, and an 8th grader at the 20th percentile, to be deemed proficient.

In South Carolina, however, 3rd graders have to score above the 60th percentile in reading and at the 71st percentile in math to reach proficiency. Eighth graders in that state must perform above the 60th percentile in reading and about the 75th percentile in math to meet the standard.

Colorado officials have been candid about setting a lower bar for the accountability purposes of NCLB than for the state accountability system. The state categorizes test-takers’ results in four groups: “advanced,” “proficient,” “partially proficient,” and “unsatisfactory.” To comply with the federal law’s AYP requirements, Colorado aims to move more students to the partially proficient category.

“We made a policy decision when NCLB came out” to use the partially proficient standard to determine AYP, said Jo O’Brien, the assistant state education commissioner for assessment. “If NCLB was really about the achievement gap, and we were putting limited federal dollars into addressing that, we wanted it to go to serving the neediest kids, those that hadn’t reached partial proficiency yet.”

The state accountability system, however, demands that schools show progress in moving more children into the proficient and advanced categories or face penalties. Ms. O’Brien noted that Colorado tends to score above the national average on the National Assessment of Educational Progress in math, reading, and science.

For South Carolina, however, the decision to set the cutoffs higher than other states has had its own drawbacks, according to state Superintendent Jim Rex.

“I think we made the right decision to have high expectations for our students,” he said. “But it has created an illusion that the gap between performance and expectations is larger in our state than other states.”

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
(Re)Focus on Dyslexia: Moving Beyond Diagnosis & Toward Transformation
Move beyond dyslexia diagnoses & focus on effective literacy instruction for ALL students. Join us to learn research-based strategies that benefit learners in PreK-8.
Content provided by EPS Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Cohesive Instruction, Connected Schools: Scale Excellence District-Wide with the Right Technology
Ensure all students receive high-quality instruction with a cohesive educational framework. Learn how to empower teachers and leverage technology.
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School Climate & Safety Webinar
How to Use Data to Combat Bullying and Enhance School Safety
Join our webinar to learn how data can help identify bullying, implement effective interventions, & foster student well-being.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal What JD Vance and Tim Walz Said About School Safety in VP Debate
Education came up in the vice presidential debate, unlike the debate between Donald Trump and Kamala Harris.
3 min read
Republican vice presidential nominee Sen. JD Vance, R-Ohio, and Democratic vice presidential nominee Minnesota Gov. Tim Walz participate in a vice presidential debate hosted by CBS News Tuesday, Oct. 1, 2024, in New York.
Republican vice presidential nominee Sen. JD Vance, R-Ohio, and Democratic vice presidential nominee Minnesota Gov. Tim Walz participate in a vice presidential debate hosted by CBS News Tuesday, Oct. 1, 2024, in New York.
Matt Rourke/AP
Federal Opinion Project 2025's Education Lead on the Controversial Policy Agenda
Here’s what the lead author of the education section in the Heritage Foundation’s proposal has to say.
9 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Federal Moms for Liberty Co-Founder Would Be 'Honored' to Be Trump's Education Secretary
The conservative education activist is "as much in the mix as anyone," according to a former GOP congressional staffer.
8 min read
Republican presidential nominee former President Donald Trump dances with Moms for Liberty co-founder Tiffany Justice during an event at the group's annual convention in Washington, Friday, Aug. 30, 2024.
Republican presidential nominee former President Donald Trump dances with Moms for Liberty co-founder Tiffany Justice during an event at the group's annual convention in Washington, Friday, Aug. 30, 2024.
Mark Schiefelbein/AP
Federal Opinion Project 2025 Might Feel New, But Its Roots Reach Back Decades
It represents the culmination of a movement to gut public education, writes Bettina L. Love.
4 min read
A group of school children is stopped from entering a bright red doorway by a large hand.
Vanessa Solis/Education Week + Getty Images