ETS Study Takes ‘Value Added’ View of NAEP

By Debra Viadero — June 17, 1998 3 min read
  • Save to favorites
  • Print

Depending on how the numbers are analyzed, U.S. students have either shown steady progress on the National Assessment of Educational Progress tests over the past 25 years or they’ve been treading water.

The conflicting interpretations emerge from a novel analysis of NAEP data by a pair of researchers from the Educational Testing Service.

Traditionally, researchers and policymakers look at trends in test scores over time to figure out whether achievement is on the rise. But the ETS researchers--Paul E. Barton and Richard J. Coley--decided to take a “value added” approach to analyzing data from NAEP. The congressionally mandated program regularly assesses a representative sample of students in grades 4, 8, and 12.

For More Information

Read “Growth in School: Achievement Gains from the Fourth to the Eighth Grade.” Or, for a copy of the report, see ordering information.

The authors looked at how much students had learned between the 4th and 8th grades, then compared that growth with gains by students over the same grade span in previous decades.

They found that despite overall rises in average test scores, American students’ learning growth in most subjects has remained basically flat since the 1970s. In mathematics, there was even a slight decline.

The reason for the discrepancy, Mr. Barton argues, is that most of the gain that students made on NAEP in recent decades showed up first when those student were 9 years old. “That just carried forward into the other grades,” said Mr. Barton, the director of the policy-information center at the Princeton, N.J.-based ETS.

Offering ‘Another Window’

But gains for that youngest testing population are as much a reflection of the richness of a child’s early home life as they are of schooling. To get a closer look at what actually happens in school, the researchers argue, it’s important to look at improvements between the 4th and 8th grades.

In both grades, for example, Maine was a top-scoring state on the NAEP math tests in 1992 and 1996. Arkansas students ranked near the bottom. Yet in each state, students improved by the same amount--52 percentile points--from 4th to 8th grade.

“Does that mean that Arkansas does as good a job as Maine?” Mr. Barton said. “Should students be scored on the basis of how much they learned or on the basis of how much they know?”

Mr. Barton said federal and state officials should report test results both ways, using regular test scores as well as value-added data.

The ETS researchers are not the first to use a value-added approach to reporting test data. Tennessee, for example, focuses on gains in achievement to help judge the effectiveness of both schools and teachers. (“A Question of Value,” May 13, 1998.)

“It’s as important to look at [the data] this way as it is to look at it another way,” Mr. Barton said.

Different Students?

One drawback to the ETS approach, however, may be that researchers can’t be sure they are testing the same students four years later, said Roy E. Truby, the executive director of the National Assessment Governing Board, which oversees NAEP. “There’s so much mobility among students.”

Also, changes to NAEP may make future comparisons impossible. The governing board has begun to move away from score scales that allow all three grade levels to be grouped on the same index so that gains could be measured from one grade to another. The most recent science test uses separate scales for each grade level.

One reason: With separate scales, test developers can include more items geared to a specific grade level. Now, to allow for comparisons over time and to measure all three grades on the same scale, the tests must repeat some questions from grade to grade.

“It’s not a slam-dunk issue. The board thought long and hard about it,” Mr. Truby said of the shift to separate scales. “But this study does show that there are some good things you can do with cross-grade scales.”

Related Tags:

A version of this article appeared in the June 17, 1998 edition of Education Week as ETS Study Takes ‘Value Added’ View of NAEP


Reading & Literacy Webinar A Roadmap to Multisensory Early Literacy Instruction: Accelerate Growth for All Students 
How can you develop key literacy skills with a diverse range of learners? Explore best practices and tips to meet the needs of all students. 
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
College & Workforce Readiness Webinar
Supporting 21st Century Skills with a Whole-Child Focus
What skills do students need to succeed in the 21st century? Explore the latest strategies to best prepare students for college, career, and life.
Content provided by Panorama Education
School & District Management K-12 Essentials Forum What Will It Take for Schools to Get Better?
Find out what educators and leaders can do to incite lasting and productive change that will make a difference in the lives of students.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Letter to the Editor We Need NAEP
The president and CEO of Knowledge Alliance responds to a recent opinion essay's criticism of the National Assessment of Educational Progress.
1 min read
Illustration of an open laptop receiving an email.
Assessment Letter to the Editor 2022 Assessment ‘Most Important’ Ever
The executive director of the National Assessment Governing Board responds to criticism of NAEP in this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.
Assessment Opinion Ignore NAEP. Better Yet, Abolish It
We’ve got to stop testing schools to death, writes Al Kingsley. National (and international) tests won't “fix” education.
Al Kingsley
5 min read
conceptual illustration of a ruler measuring a figure
Vanessa Solis/Education Week and iStock/Getty images
Assessment Opinion The Future, Present, and Past of 'the Nation's Report Card'
What lies ahead for the nation's only true barometer of the state of K-12 education?
7 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty