Federal

Report Pans How States Set the Bar

By Kathleen Kennedy Manzo — October 05, 2007 5 min read
  • Save to favorites
  • Print

The random and sometimes erratic nature of state proficiency standards makes for an assessment system that is “slipshod,” obscure, and potentially unreliable, contends an outspoken think tank’s analysis of the testing benchmarks set by 26 states.

The range of expectations—3rd grade reading proficiency can mean performing at the 6th percentile on one state’s test, but at the 62nd on another’s—spells “big trouble” for standards-based improvement efforts and the No Child Left Behind Act, concludes the report, “The Proficiency Illusion,” released last week.

“A lot of states don’t expect very much of their kids, and [standards] are enormously discrepant from state to state, subject to subject, and from grade to grade,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation, which commissioned the report. “That all adds up to say the centerpiece of standards-based reform may not be up to what we expect.”

The report from Fordham reinforces growing concerns that, despite the federal mandates and the improved reporting of student-achievement data under the NCLB law, no common yardstick exists for comparing those school improvement gains across states.

“The notion that in states with less stringent standards ... students are being declared proficient and can make their way through the system … but are not ready to compete with graduates from other states that have standards that are much more stringent is a huge problem,” said Henry Braun, a researcher at Boston College who conducted a federal study earlier this year comparing state proficiency benchmarks with those of the National Assessment of Educational Progress. (“State Tests, NAEP Often a Mismatch,” June 13, 2007.)

Mr. Braun said the study from Fordham—a strong proponent of national standards and advocate for market-based reforms—is “clearly the most detailed analysis at the state level … and is consistent with, and reinforces, other reports that deal with this issue.”

But officials from some states included in the analysis said the broad statements in the report did not accurately reflect the hard work being done to raise standards and measure results, according to Gene Wilhoit, the executive director of the Council of Chief State School Officers, based in Washington.

“They felt the report took an accusatory kind of point of view that the states had been hiding standards or had not done due diligence about their program,” he said, describing a conference call state officials had with the authors of the report prior to its release. “They were concerned whether the scale was a fair standard to use.”

Tool for Comparisons

The study was conducted by the Portland, Ore.-based Northwest Evaluation Association, which administers the Measures of Academic Progress, a computerized diagnostic assessment. The MAP assessment is given to a sufficient number of students in 26 states to allow for comparisons with state tests, the report says.

Wide Variation in Cutoff Points

States set widely differing cutoff points for determining how well a student must perform in order to qualify as “proficient” in reading or math under the No Child Left Behind Act. This table lists the estimated percentile rank needed to meet that standard in 26 states, based on the Measures of Academic Progress, a computerized diagnostic test.

Click to enlarge

SOURCE: Thomas B. Fordham Foundation

NWEA researchers John Cronin, Michael Dahlin, Deborah Adkins, and Gage Kingsbury analyzed state tests and results and estimated where the cutoff scores fall on the MAP percentile scales at each grade level for math and reading.

They found that state tests for 8th graders tend to be more difficult for those students to pass than are the 3rd grade tests for the younger pupils, even after considering the differences in developmental levels and the difficulty of the content.

That inconsistency could end up giving educators and the public a false perception of how children are doing or how they should progress through the grades, according to the study.

For example, a student who reaches proficiency on the 3rd grade test in a state with low standards for that grade may not be on track to meet a more rigorous 8th grade proficiency benchmark.

The NCLB law requires states to test students annually in grades 3-8, and once in high school, in reading and math. Schools must show adequate yearly progress in moving students toward what a state defines as proficiency in those subjects or face penalties.

The report warns that students who move to a state with higher standards generally are not prepared to meet the more rigorous expectations.

Colorado, Michigan, and Wisconsin were among the states with the lowest cutoff scores across grades 3 through 8 for both subjects, while California, Massachusetts, and South Carolina tended to set the highest benchmarks.

Changing Goal Posts

Although most states have not lowered their cutoff scores to make it easier to meet NCLB requirements that they show progress toward making sure all students are “proficient” by 2014, eight states with high standards have lowered their expectations at some grade levels since the law was passed in 2001.

The study determined that the main factors in explaining states’ gains in proficiency rates might include teaching to the test, students’ motivation to do well on the tests, and adjustments to the testing standards.

Colorado sets cutoff scores equivalent to the 6th percentile on its 3rd grade reading test—a level easy enough for 94 percent of 3rd graders nationally to meet that standard—and the 13th percentile on the 8th grade test.

In math, a Colorado 3rd grader has to score at the 7th percentile, and an 8th grader at the 20th percentile, to be deemed proficient.

In South Carolina, however, 3rd graders have to score above the 60th percentile in reading and at the 71st percentile in math to reach proficiency. Eighth graders in that state must perform above the 60th percentile in reading and about the 75th percentile in math to meet the standard.

Colorado officials have been candid about setting a lower bar for the accountability purposes of NCLB than for the state accountability system. The state categorizes test-takers’ results in four groups: “advanced,” “proficient,” “partially proficient,” and “unsatisfactory.” To comply with the federal law’s AYP requirements, Colorado aims to move more students to the partially proficient category.

“We made a policy decision when NCLB came out” to use the partially proficient standard to determine AYP, said Jo O’Brien, the assistant state education commissioner for assessment. “If NCLB was really about the achievement gap, and we were putting limited federal dollars into addressing that, we wanted it to go to serving the neediest kids, those that hadn’t reached partial proficiency yet.”

The state accountability system, however, demands that schools show progress in moving more children into the proficient and advanced categories or face penalties. Ms. O’Brien noted that Colorado tends to score above the national average on the National Assessment of Educational Progress in math, reading, and science.

For South Carolina, however, the decision to set the cutoffs higher than other states has had its own drawbacks, according to state Superintendent Jim Rex.

“I think we made the right decision to have high expectations for our students,” he said. “But it has created an illusion that the gap between performance and expectations is larger in our state than other states.”

Events

Recruitment & Retention Live Online Discussion A Seat at the Table: Chronic Teacher Shortage: Where Do We Go From Here?  
Join Peter DeWitt, Michael Fullan, and guests for expert insights into finding solutions for the teacher shortage.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Close the Gender Gap: Getting Girls Excited about STEM
Join female STEM leaders as they discuss the importance of early cheerleaders, real life role models, and female networks of support.
Content provided by Logitech
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
Mission Possible: Saving Time While Improving Student Outcomes
Learn how district leaders are maximizing instructional time and finding the best resources for student success through their MTSS framework.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Q&A Boosting 'Pathetically Low' Teacher Pay Is Top of Mind for Bernie Sanders
The progressive senator from Vermont spoke with Education Week as he prepares to chair the Senate's education committee.
6 min read
Sen. Bernie Sanders, I-Vt., talks with reporters outside the West Wing of the White House in Washington, Wednesday, Jan. 25, 2023.
Sen. Bernie Sanders, I-Vt., talks with reporters outside the West Wing of the White House in Washington, in late January.
Susan Walsh/AP
Federal What’s Behind the Push for a $60K Base Teacher Salary
When reintroduced in Congress, a bill to raise teacher salaries will include money to account for regional cost differences.
5 min read
Teachers from Seattle Public Schools picket outside Roosevelt High School on what was supposed to be the first day of classes, Wednesday, Sept. 7, 2022, in Seattle. The first day of classes at Seattle Public Schools was cancelled and teachers are on strike over issues that include pay, mental health support, and staffing ratios for special education and multilingual students.
Teachers from Seattle Public Schools picket outside Roosevelt High School on what was supposed to be the first day of classes, Wednesday, Sept. 7, 2022, in Seattle. The first day of classes at Seattle Public Schools was cancelled and teachers are on strike over issues that include pay, mental health support, and staffing ratios for special education and multilingual students.
Jason Redmond/AP
Federal Teachers Shouldn't Have to Drive Ubers on the Side, Education Secretary Says
In a speech on priorities for the year, U.S. Education Secretary Miguel Cardona said teachers should be paid competitive salaries.
5 min read
U.S. Secretary of Education Miguel Cardona delivers a speech during the “Raise the Bar: Lead the World” event in Washington, D.C., on Jan. 24, 2023.
U.S. Secretary of Education Miguel Cardona delivers a speech during the “Raise the Bar: Lead the World” event in Washington, D.C., on Jan. 24, 2023.
Sam Mallon/Education Week
Federal A Chaotic Start to a New Congress: What Educators Need to Know
A new slate of lawmakers will have the chance to influence federal education policy in the 118th Congress.
4 min read
Rep. Kevin McCarthy, R-Calif., talks on the House floor after the first vote for House Speaker when he did not receive enough votes to be elected during opening day of the 118th Congress at the U.S. Capitol, Tuesday, Jan 3, 2023, in Washington.
Rep. Kevin McCarthy, R-Calif., talks on the floor of the U.S. House of Representatives on Jan. 3 following the first round of voting for House Speaker. McCarthy fell short of enough votes to be elected speaker in three rounds of voting on opening day of the 118th Congress at the U.S. Capitol.
Andrew Harnik/AP