Federal

State Tests, NAEP Often a Mismatch

By Sean Cavanagh — June 07, 2007 5 min read
  • Save to favorites
  • Print

Many of the states that claim to have large shares of their students reaching proficiency in reading and mathematics under the No Child Left Behind Act have set less stringent standards for meeting that threshold than lower-performing states, a new federal study finds.

The study drew an immediate and strong reaction from many public officials and education advocates, who said it laid bare states’ vastly divergent standards for testing students.

The report judges states’ reading and math tests against a common yardstick: the proficiency standards used by the National Assessment of Educational Progress, often referred to as “the nation’s report card.”

Released last week by the National Center for Education Statistics, the analysis appears to back up the suspicions of those who have cast a skeptical eye on state data showing high percentages of students reaching the “proficient” level in reading and math.

See Also

For more stories on this topic see our annual series, NCLB: Taking Root.

But researchers who were asked by the Council of Chief State School Officers to review the study’s methodology cited what they see as flaws in comparing two dissimilar sets of exams: NAEP and those administered by states.

Even so, U.S. Secretary of Education Margaret Spellings called it “sobering news” as the nation seeks to raise academic demands on students.

States “must do their part by setting high standards and expectations,” she said in a statement. “I hope this report will be a catalyst for positive change.”

The study was issued June 7, two days after a separate report by an education policy group showing that student scores on state tests have risen since the enactment of the No Child Left Behind Act, which President Bush signed into law in January 2002. (“State Tests Show Gains Since NCLB,” June 6, 2007.)

‘Absolutely Huge’

The NCES study, “Mapping 2005 State Proficiency Standards Onto the NAEP Scales,” doesn’t attempt to compare the difficulty of individual state tests in reading and math, or how they compare with the national assessment, which is given to representative samples of students.

Instead, it compares where states set minimum scores for determining whether students are proficient, under the mandates of the No Child Left Behind Act, against the bar set by NAEP on the 4th and 8th grade reading and math tests. The study uses NAEP data from the 2004-05 administration of the assessment and gives states a “NAEP score equivalent.”

Under the NCLB law, states are required to test students annually in reading and math in grades 3-8 and once in high school, and report the percentages of students achieving proficiency on those exams. Schools must make adequate yearly progress in those subjects or face increasingly stiff penalties.

Stacking Up

A new study shows how state proficiency standards rank against those of NAEP. Of 36 states taking part in 8th grade math, Missouri’s standard ranked highest, while Tennessee’s was lowest.

BRIC ARCHIVE

SOURCE: National Center for Education Statistics

The study found that state tests varied greatly in judging students as proficient—between 60 and 80 points—when placed on the NAEP 500-point scale.

“That’s absolutely huge,” said Eric A. Hanushek, a senior fellow at the Hoover Institution at Stanford University who co-wrote the report on student tests for the Center on Education Policy. “You’re talking about very large differences in how states are measuring the performance of kids.”

In 4th grade reading, the NCES study found wide variation in states’ standards for proficiency when judged against the NAEP scale.

Massachusetts, South Carolina, Wyoming, Arkansas, and Connecticut, in that order, had the five highest “NAEP score equivalents.” In other words, Massachusetts’ proficiency standard matched the NAEP standard for 4th grade reading better than any other state’s. Mississippi had the worst score equivalents in that subject and grade, followed by Tennessee, Georgia, Alaska, and Oklahoma.

Call to Candidates

Reaction to the federal study came from many fronts.

The Education Trust, an influential Washington group that advocates high standards and vigorous efforts to help disadvantaged children meet them, said “students and their parents are being given a false sense of promise,” which is that “children are being prepared to meet the real-world challenges of college and careers.”

The response took on a political, but bipartisan, dimension when Ken Mehlman, the former chairman of the Republican National Committee, and Roy R. Romer, a former general chairman of its Democratic counterpart, issued a joint statement urging the 2008 presidential candidates to make education a priority in their campaign platforms.

But two researchers who reviewed the study’s methods for the Council of Chief State School Officers questioned the reliability of linking NAEP with state exams, in two separate papers.

Andrew Ho, of the University of Iowa, and Edward Haertel of Stanford University, cited the “large body of literature” pointing to differences in the purposes and design of state tests and NAEP. As many researchers have noted, states craft tests and set achievement levels for them based on where most students are likely to score, and to set realistic goals for improvement, they said. NAEP tests, by contrast, serve “lofty, long-term goals,” they write, with much higher achievement levels.

One of the authors of the NCES study, Henry Braun, said in an interview that even though NAEP is “an imperfect” standard for judging state proficiency standards, it is still a useful one.

“We need to ask if [this] kind of variation is in the interest of the nation and its schools,” said Mr. Braun, a professor of education at Boston College.

Changes Afoot

Officials in Tennessee, which scored poorly in the nces study, were not surprised by the findings, said Rachel Woods, a spokeswoman for the state education department. State leaders already recognize its bar for judging proficiency is too low, and they are revising Tennessee’s content standards and exams to fix it, she said.

“This is stuff we already figured out two years ago, and we’re working on it,” Ms. Woods said.

Jim Rex, the superintendent of schools in South Carolina, among the highest-scoring states in the NCES study, said the result might help the public in his state better understand why the number of students reaching the proficient level on state exams is relatively low.

“We’ve had trouble in this state that proficiency scores were low because our standards were so high,” Mr. Rex said. “I’d like to see this information given to parents annually in all states. They can draw the conclusions they want to.”

A version of this article appeared in the June 13, 2007 edition of Education Week

Events

Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Accelerate Learning with Project-Based Learning
Earlier this year, the George Lucas Educational Foundation released four new studies highlighting how project-based learning (PBL) helps accelerate student learning—across age groups, multiple disciplines, and different socio-economic statuses. With this year’s emphasis on unfinished
Content provided by SmartLab Learning
School & District Management Live Online Discussion Principal Overload: How to Manage Anxiety, Stress, and Tough Decisions
According to recent surveys, more than 40 percent of principals are considering leaving their jobs. With the pandemic, running a school building has become even more complicated, and principals' workloads continue to grow. f we

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal California, Florida, and Other States Waiting on Green Light for Their COVID Relief Plans
The list of states with Ed. Dept. approval for their American Rescue Plan blueprints is growing steadily, but two big states aren't on it.
4 min read
Image shows an illustration of money providing relief against coronavirus.
DigitalVision Vectors/iStock/Getty
Federal What Is a School Shooting? Members of Congress Seek a Federal Definition, Reliable Data
A new bill would direct federal departments to track data related to school shootings, a term for which there is no federal definition.
Daniela Altimari, Hartford Courant
4 min read
Police respond to the scene of a shooting at Heritage High School in Newport News, Va., on Saturday Sept. 20, 2021. Newport News police Chief Steve Drew said two students were shot and taken to the hospital and neither injury was thought to be life-threatening. The chief said authorities believe the suspect and victims knew one another but did not provide details.
Police respond to the scene of a shooting at Heritage High School in Newport News, Va., on Saturday Sept. 20, 2021.
John C. Clark/AP Photo
Federal Is the Justice Dept. Silencing Parents or Stepping Up to Protect Educators?
Merrick Garland's move to use the FBI to help protect school officials from violence and harassment has drawn anger and praise.
5 min read
Sen. Josh Hawley, R-Mo., speaks during the Senate Judiciary Committee hearing to examine Texas's abortion law, Wednesday, Sept. 29, 2021 on Capitol Hill in Washington.
Sen. Josh Hawley, R-Mo., speaks during the Senate Judiciary Committee hearing to examine Texas's abortion law, Wednesday, Sept. 29, 2021 on Capitol Hill in Washington.
Tom Williams/Pool via AP
Federal Don't Use Federal COVID Aid to Undermine School Mask Rules, U.S. Treasury Tells Governor
Arizona Gov. Doug Ducey violated the intent of COVID aid programs by using them to discourage school mask mandates, an agency letter says.
2 min read
Arizona Gov. Doug Ducey speaks at the Arizona Capitol in Phoenix on Nov. 30, 2020. A program announced by Arizona's Republican governor last month to give private school vouchers to students whose parents object to school mask requirements has seen a surge of applications, with twice as many either completed or started than can be funded with the $10 million in federal coronavirus relief cash he earmarked for the program.
A program announced by Arizona's Republican Gov. Doug Ducey in September earmarks federal money to give private school vouchers to students whose parents object to public school mask requirements.
Ross D. Franklin/AP