Federal

Report Pans How States Set the Bar

By Kathleen Kennedy Manzo — October 05, 2007 5 min read
  • Save to favorites
  • Print

The random and sometimes erratic nature of state proficiency standards makes for an assessment system that is “slipshod,” obscure, and potentially unreliable, contends an outspoken think tank’s analysis of the testing benchmarks set by 26 states.

The range of expectations—3rd grade reading proficiency can mean performing at the 6th percentile on one state’s test, but at the 62nd on another’s—spells “big trouble” for standards-based improvement efforts and the No Child Left Behind Act, concludes the report, “The Proficiency Illusion,” released last week.

“A lot of states don’t expect very much of their kids, and [standards] are enormously discrepant from state to state, subject to subject, and from grade to grade,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation, which commissioned the report. “That all adds up to say the centerpiece of standards-based reform may not be up to what we expect.”

The report from Fordham reinforces growing concerns that, despite the federal mandates and the improved reporting of student-achievement data under the NCLB law, no common yardstick exists for comparing those school improvement gains across states.

“The notion that in states with less stringent standards ... students are being declared proficient and can make their way through the system … but are not ready to compete with graduates from other states that have standards that are much more stringent is a huge problem,” said Henry Braun, a researcher at Boston College who conducted a federal study earlier this year comparing state proficiency benchmarks with those of the National Assessment of Educational Progress. (“State Tests, NAEP Often a Mismatch,” June 13, 2007.)

Mr. Braun said the study from Fordham—a strong proponent of national standards and advocate for market-based reforms—is “clearly the most detailed analysis at the state level … and is consistent with, and reinforces, other reports that deal with this issue.”

But officials from some states included in the analysis said the broad statements in the report did not accurately reflect the hard work being done to raise standards and measure results, according to Gene Wilhoit, the executive director of the Council of Chief State School Officers, based in Washington.

“They felt the report took an accusatory kind of point of view that the states had been hiding standards or had not done due diligence about their program,” he said, describing a conference call state officials had with the authors of the report prior to its release. “They were concerned whether the scale was a fair standard to use.”

Tool for Comparisons

The study was conducted by the Portland, Ore.-based Northwest Evaluation Association, which administers the Measures of Academic Progress, a computerized diagnostic assessment. The MAP assessment is given to a sufficient number of students in 26 states to allow for comparisons with state tests, the report says.

Wide Variation in Cutoff Points

States set widely differing cutoff points for determining how well a student must perform in order to qualify as “proficient” in reading or math under the No Child Left Behind Act. This table lists the estimated percentile rank needed to meet that standard in 26 states, based on the Measures of Academic Progress, a computerized diagnostic test.

Click to enlarge

SOURCE: Thomas B. Fordham Foundation

NWEA researchers John Cronin, Michael Dahlin, Deborah Adkins, and Gage Kingsbury analyzed state tests and results and estimated where the cutoff scores fall on the MAP percentile scales at each grade level for math and reading.

They found that state tests for 8th graders tend to be more difficult for those students to pass than are the 3rd grade tests for the younger pupils, even after considering the differences in developmental levels and the difficulty of the content.

That inconsistency could end up giving educators and the public a false perception of how children are doing or how they should progress through the grades, according to the study.

For example, a student who reaches proficiency on the 3rd grade test in a state with low standards for that grade may not be on track to meet a more rigorous 8th grade proficiency benchmark.

The NCLB law requires states to test students annually in grades 3-8, and once in high school, in reading and math. Schools must show adequate yearly progress in moving students toward what a state defines as proficiency in those subjects or face penalties.

The report warns that students who move to a state with higher standards generally are not prepared to meet the more rigorous expectations.

Colorado, Michigan, and Wisconsin were among the states with the lowest cutoff scores across grades 3 through 8 for both subjects, while California, Massachusetts, and South Carolina tended to set the highest benchmarks.

Changing Goal Posts

Although most states have not lowered their cutoff scores to make it easier to meet NCLB requirements that they show progress toward making sure all students are “proficient” by 2014, eight states with high standards have lowered their expectations at some grade levels since the law was passed in 2001.

The study determined that the main factors in explaining states’ gains in proficiency rates might include teaching to the test, students’ motivation to do well on the tests, and adjustments to the testing standards.

Colorado sets cutoff scores equivalent to the 6th percentile on its 3rd grade reading test—a level easy enough for 94 percent of 3rd graders nationally to meet that standard—and the 13th percentile on the 8th grade test.

In math, a Colorado 3rd grader has to score at the 7th percentile, and an 8th grader at the 20th percentile, to be deemed proficient.

In South Carolina, however, 3rd graders have to score above the 60th percentile in reading and at the 71st percentile in math to reach proficiency. Eighth graders in that state must perform above the 60th percentile in reading and about the 75th percentile in math to meet the standard.

Colorado officials have been candid about setting a lower bar for the accountability purposes of NCLB than for the state accountability system. The state categorizes test-takers’ results in four groups: “advanced,” “proficient,” “partially proficient,” and “unsatisfactory.” To comply with the federal law’s AYP requirements, Colorado aims to move more students to the partially proficient category.

“We made a policy decision when NCLB came out” to use the partially proficient standard to determine AYP, said Jo O’Brien, the assistant state education commissioner for assessment. “If NCLB was really about the achievement gap, and we were putting limited federal dollars into addressing that, we wanted it to go to serving the neediest kids, those that hadn’t reached partial proficiency yet.”

The state accountability system, however, demands that schools show progress in moving more children into the proficient and advanced categories or face penalties. Ms. O’Brien noted that Colorado tends to score above the national average on the National Assessment of Educational Progress in math, reading, and science.

For South Carolina, however, the decision to set the cutoffs higher than other states has had its own drawbacks, according to state Superintendent Jim Rex.

“I think we made the right decision to have high expectations for our students,” he said. “But it has created an illusion that the gap between performance and expectations is larger in our state than other states.”

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Biden Calls for Teacher Pay Raises, Expanded Pre-K in State of the Union
President Joe Biden highlighted a number of his education priorities in a high-stakes speech as he seeks a second term.
5 min read
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
Shawn Thew/Pool via AP
Federal Low-Performing Schools Are Left to Languish by Districts and States, Watchdog Finds
Fewer than half of district plans for improving struggling schools meet bare minimum requirements.
11 min read
A group of silhouettes looks across a grid with a public school on the other side.
Vanessa Solis/Education Week via Canva
Federal Biden Admin. Says New K-12 Agenda Tackles Absenteeism, Tutoring, Extended Learning
The White House unveiled a set of K-12 priorities at the start of an election year.
4 min read
U.S. Education Secretary Miguel Cardona participates in a roundtable discussion with students from Dartmouth College on Jan. 10, 2024, on the school's campus, in Hanover, N.H.
U.S. Education Secretary Miguel Cardona participates in a roundtable discussion with students from Dartmouth College on Jan. 10, 2024, on the school's campus, in Hanover, N.H.
Steven Senne/AP
Federal Lawmakers Want to Reauthorize a Major Education Research Law. What Stands in the Way?
Lawmakers have tried and failed to reauthorize the Education Sciences Reform Act over the past nearly two decades.
7 min read
Sen. Bill Cassidy, R-La., left, joins Senate Health, Education, Labor and Pensions Committee Chairman Bernie Sanders, I-Vt., left, as Starbucks founder Howard Schultz answers questions about the company's actions during an ongoing employee unionizing campaign, at the Capitol in Washington, on March 29, 2023.
Sen. Bill Cassidy, R-La., left, joins Senate Health, Education, Labor and Pensions Committee Chairman Bernie Sanders, I-Vt., left, at the Capitol in Washington, on March 29, 2023. The two lawmakers sponsored a bill to reauthorize the Education Sciences Reform Act.
J. Scott Applewhite/AP