ETS Study Takes ‘Value Added’ View of NAEP

By Debra Viadero — June 17, 1998 3 min read
  • Save to favorites
  • Print

Depending on how the numbers are analyzed, U.S. students have either shown steady progress on the National Assessment of Educational Progress tests over the past 25 years or they’ve been treading water.

The conflicting interpretations emerge from a novel analysis of NAEP data by a pair of researchers from the Educational Testing Service.

Traditionally, researchers and policymakers look at trends in test scores over time to figure out whether achievement is on the rise. But the ETS researchers--Paul E. Barton and Richard J. Coley--decided to take a “value added” approach to analyzing data from NAEP. The congressionally mandated program regularly assesses a representative sample of students in grades 4, 8, and 12.

For More Information

Read “Growth in School: Achievement Gains from the Fourth to the Eighth Grade.” Or, for a copy of the report, see ordering information.

The authors looked at how much students had learned between the 4th and 8th grades, then compared that growth with gains by students over the same grade span in previous decades.

They found that despite overall rises in average test scores, American students’ learning growth in most subjects has remained basically flat since the 1970s. In mathematics, there was even a slight decline.

The reason for the discrepancy, Mr. Barton argues, is that most of the gain that students made on NAEP in recent decades showed up first when those student were 9 years old. “That just carried forward into the other grades,” said Mr. Barton, the director of the policy-information center at the Princeton, N.J.-based ETS.

Offering ‘Another Window’

But gains for that youngest testing population are as much a reflection of the richness of a child’s early home life as they are of schooling. To get a closer look at what actually happens in school, the researchers argue, it’s important to look at improvements between the 4th and 8th grades.

In both grades, for example, Maine was a top-scoring state on the NAEP math tests in 1992 and 1996. Arkansas students ranked near the bottom. Yet in each state, students improved by the same amount--52 percentile points--from 4th to 8th grade.

“Does that mean that Arkansas does as good a job as Maine?” Mr. Barton said. “Should students be scored on the basis of how much they learned or on the basis of how much they know?”

Mr. Barton said federal and state officials should report test results both ways, using regular test scores as well as value-added data.

The ETS researchers are not the first to use a value-added approach to reporting test data. Tennessee, for example, focuses on gains in achievement to help judge the effectiveness of both schools and teachers. (“A Question of Value,” May 13, 1998.)

“It’s as important to look at [the data] this way as it is to look at it another way,” Mr. Barton said.

Different Students?

One drawback to the ETS approach, however, may be that researchers can’t be sure they are testing the same students four years later, said Roy E. Truby, the executive director of the National Assessment Governing Board, which oversees NAEP. “There’s so much mobility among students.”

Also, changes to NAEP may make future comparisons impossible. The governing board has begun to move away from score scales that allow all three grade levels to be grouped on the same index so that gains could be measured from one grade to another. The most recent science test uses separate scales for each grade level.

One reason: With separate scales, test developers can include more items geared to a specific grade level. Now, to allow for comparisons over time and to measure all three grades on the same scale, the tests must repeat some questions from grade to grade.

“It’s not a slam-dunk issue. The board thought long and hard about it,” Mr. Truby said of the shift to separate scales. “But this study does show that there are some good things you can do with cross-grade scales.”

Related Tags:

A version of this article appeared in the June 17, 1998 edition of Education Week as ETS Study Takes ‘Value Added’ View of NAEP


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Start Strong With Solid SEL Implementation: Success Strategies for the New School Year
Join Satchel Pulse to learn why implementing a solid SEL program at the beginning of the year will deliver maximum impact to your students.
Content provided by Satchel Pulse
Teaching Live Online Discussion Seat at the Table: How Can We Help Students Feel Connected to School?
Get strategies for your struggles with student engagement. Bring questions for our expert panel. Help students recover the joy of learning.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Science Webinar
Real-World Problem Solving: How Invention Education Drives Student Learning
Hear from student inventors and K-12 teachers about how invention education enhances learning, opens minds, and preps students for the future.
Content provided by The Lemelson Foundation

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion The Future, Present, and Past of 'the Nation's Report Card'
What lies ahead for the nation's only true barometer of the state of K-12 education?
7 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Assessment The 'Nation's Report Card' Is Getting an Overhaul: 5 Things to Know
The leaders of NAEP have big plans for making the test more nimble, flexible, and useful.
9 min read
Image of a bank of computers in a library.
Assessment Opinion What the Digital SAT Will Mean for Students and Educators
The college-admissions test will be fully digital by 2024. Priscilla Rodriguez from the College Board discusses the change.
6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Assessment Opinion Searching for Common Ground: What Makes a Good Test?
Rick Hess and USC Dean Pedro Noguera discuss standardized testing—what it’s for, where it’s gone wrong, and how to improve it.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty