California Study Questions Validity of Gains Under NCLB

By Sean Cavanagh — July 11, 2006 4 min read
  • Save to favorites
  • Print

Student progress in reading has stagnated and in some cases declined since the passage of the No Child Left Behind Act, says a study released last week that examines test scores in a dozen states and finds that their gains are often exaggerated as a result of overly easy exams.

The report, issued by researchers at three universities in California, compares 4th graders’ reading and math scores on individual, state-designed assessments with those on the National Assessment of Educational Progress.

“Is the No Child Left Behind Act Working? The Reliability of How States Track Achievement” is posted by Policy Analysis for California Education.

The study points to the disparities between the percentage of students many of those states deem to be at least “proficient” in reading and math and those achieving that distinction on NAEP.

The study also found that states made stronger average annual gains in reading on the NAEP in the decade before the No Child Left Behind law took effect, in 2002.

Which Scores to Trust

States were more successful in sustaining their progress in math after the far-reaching, 4½-year-old law took effect, the study found, though their NAEP gains were more modest than those reported on individual state tests.

The report, “Is the No Child Left Behind Act Working? The Reliability of How States Track Achievement,” was released by Policy Analysis for California Education, or PACE, a research center affiliated with the University of California’s Berkeley and Davis campuses and Stanford University.

“Testing was sold to parents and the public as a way to tell whether student achievement was going up or down,” Bruce Fuller, a UC- Berkeley professor of education and one of the report’s authors, said in an interview. “Parents and policymakers and journalists should not trust state test-score results. They should place them alongside national test-score results.”

Valerie L. Smith, a spokeswoman for the U.S. Department of Education, disputed the report’s findings, noting recent progress in nationwide NAEP reading scores, especially among minority students. She noted that No Child Left Behind requires states to participate in NAEP as a way to ensure that states are held to account.

“The fact that state scores have risen more quickly than NAEP scores does nothing to diminish the gains we have seen on both state and NAEP [tests],” Ms. Smith said in a statement.

States craft their own tests in reading and math, which, under the provisions of the federal law, must be given annually in grades 3-8, and at least once in high school. Schools and districts that do not make adequate yearly progress face penalties.

How Proficient?

The content and rigor of state tests vary widely, however. To get a true sense of student academic progress across states, federal officials and researchers often turn to the congressionally mandated NAEP, which bills itself as the only nationally representative and continuing sample of student knowledge in various subjects. While almost all students are required to take their states’ assessments, NAEP, often called “the nation’s report card,” tests only a sample of students within states.

State tests and the national assessment judge student scores at different achievement levels. But the PACE study concludes that states set the bar much lower than NAEP does in determining whether students reach at least the “proficient” level.

On average, the proportion of 4th graders considered at least proficient in reading was 35 percentage points higher on the 12 states’ tests than on NAEP. In math, the proportion of students who reached the proficient level was 36 percentage points higher on the state exams than on NAEP.

Some states’ results suggested their achievement levels are relatively close to the NAEP standard, the study found. In Massachusetts, the proportion of 4th graders reaching proficiency in reading was only 10 percentage points higher than that of NAEP, and the gap in math was only 1 point.

One of the states in the study with the largest disparity between its scores and the NAEP is Texas. The percentage of 4th graders considered at least proficient in reading on the Texas state assessment was 55 percentage points higher than on NAEP, and in math, it was 51 points higher than on the national assessment.

Deborah Graves Ratcliffe, a spokeswoman for the Texas Education Agency, said Texas students were more likely to achieve a proficient score on the state assessment because standards and classroom lessons were designed to prepare them for that exam. NAEP is designed independently, rather than adhering to the math and reading content offered in any single state.

“We know our test is based on our standards and does a good job of testing what’s taught in our schools,” Ms. Ratcliffe said. “NAEP is not on the radar screen for a lot of our students.”

A version of this article appeared in the July 12, 2006 edition of Education Week as California Study Questions Validity of Gains Under NCLB


Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Building Teacher Capacity for Social-Emotional Learning
Set goals that support adult well-being and social-emotional learning: register today!

Content provided by Panorama
Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment State Test Results Are In. Are They Useless?
While states, districts, and schools pore over data from spring 2021 tests, experts urge caution over how to interpret and use the results.
9 min read
FILE - In this Jan. 17, 2016 file photo, a sign is seen at the entrance to a hall for a college test preparation class in Bethesda, Md. The $380 million test coaching industry is facing competition from free or low-cost alternatives in what their founders hope will make the process of applying to college more equitable. Such innovations are also raising questions about the relevance and the fairness of relying on standardized tests in admissions process.
A sign is posted at the entrance to a hall for a test-preparation class. Assessment experts say educators should use data from spring 2021 tests with caution.
Alex Brandon/AP
Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
Proven Techniques for Assessing Students with Technology
Dr. Doug Fisher’s proven assessment techniques help your students become active learners and increase their chances for higher learning g...
Content provided by Achieve3000
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP