Special Report
Assessment

Executive Summary: Below the Surface

An analysis of test data finds the results both heartening and sobering.
January 03, 2006 7 min read
  • Save to favorites
  • Print

State efforts to carry out standards-based education over the past decade have a positive, but modest, relationship with gains in student achievement on the National Assessment of Educational Progress, according to this year’s edition of Quality Counts.

The 10th annual report by Education Week examines the progress that states have made on a core set of policy indicators related to standards-based education over the past decade. It also explores the relationship between changes on those indicators and gains in student learning, as measured by NAEP reading and mathematics tests in grades 4 and 8 from 1996 to 2005.

The report does not examine the more recent impact of the federal No Child Left Behind Act, which President Bush signed into law in 2002.

The results are at once heartening and sobering.

They’re heartening because when looked at over more than a decade, student achievement has gotten better, particularly in mathematics and particularly for low-income and minority students. An increasing number of states also have embraced a standards-based-education framework, with some of the earliest and most ardent adopters of standards-based accountability systems making some of the most progress in student achievement.

Nationally, NAEP scores in 4th grade math have increased by 18.5 points on a 500-point scale, or nearly two grade levels, since 1992, near the start of the standards movement.

See Also

For the first time, the Editorial Projects in Education Research Center also has produced detailed state-by-state reports on how states have performed on this year’s indicators and the progress they’ve made over time.

See the state highlights reports, which replace and expand on the state summaries that appeared in previous print editions of Quality Counts.

Even more encouraging are the gains for black and Hispanic 4th graders: 27.7 points and 24.2 points, respectively. One way to think about those gains is that if the scores for white students had not also improved, the advances would have been enough to shrink the black-white achievement gap that existed in 1992 by 80 percent, and the gap between Hispanics and non-Hispanic whites by 94 percent, virtually closing the gap between those groups in 4th grade math.

The scores for low-income students, which NAEP began reporting in 1996, basically mirror the average national increases.

In math, improvement has come at the “basic” and “proficient” levels on NAEP, both for students overall and for black, Hispanic, and low-income youngsters. Those gains are mirrored across most states, according to special analyses of NAEP scale scores conducted for Quality Counts 2006 by the Educational Testing Service, which looked at changes in state performance between 1992 and 2005. Of the 41 states and the District of Columbia with data, all made significant progress in 4th grade math achievement, and all but two states made significant progress in 8th grade math achievement during that period.

It’s hard to be as sanguine about reading. The national average in reading barely budged from 1992 to 2005, inching up just 2 points in both grades 4 and 8. But, even here, somewhat better news lies beneath the surface. The scores for black, Hispanic, and low-income youngsters in 4th grade reading increased at nearly triple the national average, or about two-thirds of a grade level.

Nationally, the achievement gaps narrowed somewhat for all groups in reading and math and at both grade levels—except between black and white students in 8th grade reading. The gap-closing was significant between black and white students in math at both grades, and between Hispanic and non-Hispanic white students in 4th grade math.

Progress is even more mixed at the state level, according to the ETS analysis, although the picture is complicated by the fact that many states either did not take part in the state-level NAEP during the periods examined or did not have enough poor or minority students in the NAEP samples to permit valid comparisons of change over time.

The 2006 report highlights individual states—including Delaware, Massachusetts, New York, North Carolina, and Texas—whose progress stands out over the past decade, with in-depth profiles that explore what might explain such changes.

The more controversial question is whether state policies over the past decade have made any difference in raising student achievement.

Quality Counts at 10: A Decade of Standards-Based Education tracks state policy initiatives over the past decade in four core areas—standards, assessments, accountability, and efforts to improve teacher quality—based on 24 specific indicators.

The report documents a steady increase in the number of states embracing state content standards and tests linked to those standards between 1997 and 2005. Fewer states adopted all of the accountability and teacher-quality indicators tracked for the report during the same period.

To examine the relationship between standards-based education and gains on NAEP, the Editorial Projects in Education Research Center conducted a series of regression analyses for Quality Counts 2006. The independent variable was changes in the strength of states’ standards-based policies between 1997 and 2005. The dependent variable was changes in NAEP math achievement between 1996 and 2005, and in NAEP reading achievement between 1998 and 2005.

The center conducted separate analyses for math and reading in grades 4 and 8. To avoid biasing the results, the analyses controlled for states’ initial NAEP performance at the start of the period, and for the initial strength of their standards-based policies.

One way in which the research center’s analysis differs from other studies is that it considered the overall effect of states’ standards-based policies—across the areas of standards, assessments, accountability, and efforts to improve teacher quality—and looked at which of those policy areas, relatively speaking, independently contributed the most to changes in achievement.

Initially, the analyses found a moderate positive relationship between states’ overall embrace of standards-based education and gains in student math achievement. The researchers observed a slight negative relationship for reading. Further analyses exploring the relative contribution of the four major policy components revealed that the implementation of policies related to teacher quality related negatively to achievement gains in both reading and math.

In a second analysis, the researchers eliminated the teacher-quality policies from the overall measure of standards-based-reform implementation in order to focus specifically on the contributions of policies related to standards, assessments, and accountability. Once teacher quality was taken out of the picture, the relationship between states’ efforts to implement standards-based reforms and gains in student achievement became much stronger.

Improvement for math in grades 4 and 8 became statistically significant, while more modest, but positive, effects emerged for reading.

Preliminary analyses also found no relationship between state resource and equity indicators and student-achievement gains, after controlling for states’ initial NAEP performance.

Such exploratory analyses have their limits.

Gains on NAEP may fall short of measuring changes in student achievement at the state level, in part because NAEP tests are given only to a sample of students in each state, do not carry any stakes for individual students or schools, and may not completely match state content standards and priorities. States also vary in the specific policies they employ, when they put them into effect, and how consistently and coherently they carry them out. From a statistical perspective, the analysis also relies on a small sample, the 50 states. That fact provides another reason for caution in interpreting the results.

Despite those caveats, the time and energy put into standards-based education over the past decade suggest it’s appropriate to examine the effects of state policies as a way to help inform state practices in the future. The results suggest that while standards-based education and its implementation are far from perfect, they can contribute to improvement in American schools.

As part of this 10-year retrospective on standards-based education, Education Week also invited five prominent policy observers to contribute their personal views to Quality Counts on what standards-based policies have accomplished so far, and what the next phase of improvement steps should be.

As is true every year, the 2006 report also tracks student achievement across the 50 states and the District of Columbia, and charts progress on states’ education systems in four areas: standards and accountability, efforts to improve teacher quality, school climate, and school resources and the equity of school finance systems. States averaged a C+ across the graded categories this year.

Related Tags:

In March 2024, Education Week announced the end of the Quality Counts report after 25 years of serving as a comprehensive K-12 education scorecard. In response to new challenges and a shifting landscape, we are refocusing our efforts on research and analysis to better serve the K-12 community. For more information, please go here for the full context or learn more about the EdWeek Research Center.

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment As They Revamp Grading, Districts Try to Improve Consistency, Prevent Inflation
Districts have embraced bold changes to make grading systems more consistent, but some say they've inflated grades and sent mixed signals.
10 min read
Close crop of a teacher's hands grading a stack of papers with a red marker.
E+
Assessment Opinion What's the Best Way to Grade Students? Teachers Weigh In
There are many ways to make grading a better, more productive experience for students. Here are a few.
14 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Spotlight Spotlight on Assessment
This Spotlight will help you evaluate effective ways to offer students feedback, learn how to improve assessments for ELs, and more.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Understanding 'Through-Year' Assessment: What Everyone Should Know
This is a once-in-a-generation opportunity to reconsider our assessment systems. Discover a fresh approach with Through-Year Assessment.
Content provided by New Meridian