ETS Study Takes ‘Value Added’ View of NAEP

By Debra Viadero — June 17, 1998 3 min read

Depending on how the numbers are analyzed, U.S. students have either shown steady progress on the National Assessment of Educational Progress tests over the past 25 years or they’ve been treading water.

The conflicting interpretations emerge from a novel analysis of NAEP data by a pair of researchers from the Educational Testing Service.

Traditionally, researchers and policymakers look at trends in test scores over time to figure out whether achievement is on the rise. But the ETS researchers--Paul E. Barton and Richard J. Coley--decided to take a “value added” approach to analyzing data from NAEP. The congressionally mandated program regularly assesses a representative sample of students in grades 4, 8, and 12.

For More Information

Read “Growth in School: Achievement Gains from the Fourth to the Eighth Grade.” Or, for a copy of the report, see ordering information.

The authors looked at how much students had learned between the 4th and 8th grades, then compared that growth with gains by students over the same grade span in previous decades.

They found that despite overall rises in average test scores, American students’ learning growth in most subjects has remained basically flat since the 1970s. In mathematics, there was even a slight decline.

The reason for the discrepancy, Mr. Barton argues, is that most of the gain that students made on NAEP in recent decades showed up first when those student were 9 years old. “That just carried forward into the other grades,” said Mr. Barton, the director of the policy-information center at the Princeton, N.J.-based ETS.

Offering ‘Another Window’

But gains for that youngest testing population are as much a reflection of the richness of a child’s early home life as they are of schooling. To get a closer look at what actually happens in school, the researchers argue, it’s important to look at improvements between the 4th and 8th grades.

In both grades, for example, Maine was a top-scoring state on the NAEP math tests in 1992 and 1996. Arkansas students ranked near the bottom. Yet in each state, students improved by the same amount--52 percentile points--from 4th to 8th grade.

“Does that mean that Arkansas does as good a job as Maine?” Mr. Barton said. “Should students be scored on the basis of how much they learned or on the basis of how much they know?”

Mr. Barton said federal and state officials should report test results both ways, using regular test scores as well as value-added data.

The ETS researchers are not the first to use a value-added approach to reporting test data. Tennessee, for example, focuses on gains in achievement to help judge the effectiveness of both schools and teachers. (“A Question of Value,” May 13, 1998.)

“It’s as important to look at [the data] this way as it is to look at it another way,” Mr. Barton said.

Different Students?

One drawback to the ETS approach, however, may be that researchers can’t be sure they are testing the same students four years later, said Roy E. Truby, the executive director of the National Assessment Governing Board, which oversees NAEP. “There’s so much mobility among students.”

Also, changes to NAEP may make future comparisons impossible. The governing board has begun to move away from score scales that allow all three grade levels to be grouped on the same index so that gains could be measured from one grade to another. The most recent science test uses separate scales for each grade level.

One reason: With separate scales, test developers can include more items geared to a specific grade level. Now, to allow for comparisons over time and to measure all three grades on the same scale, the tests must repeat some questions from grade to grade.

“It’s not a slam-dunk issue. The board thought long and hard about it,” Mr. Truby said of the shift to separate scales. “But this study does show that there are some good things you can do with cross-grade scales.”

Related Tags:

A version of this article appeared in the June 17, 1998 edition of Education Week as ETS Study Takes ‘Value Added’ View of NAEP


Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Engaging Young Students to Accelerate Math Learning
Join learning scientists and inspiring district leaders, for a timely panel discussion addressing a school district’s approach to doubling and tripling Math gains during Covid. What started as a goal to address learning gaps in
Content provided by Age of Learning & Digital Promise, Harlingen CISD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
How to Power Your Curriculum With Digital Books
Register for this can’t miss session looking at best practices for utilizing digital books to support their curriculum.
Content provided by OverDrive

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
Facing the Future Together: Digital Innovative Solutions
Join us to discuss how digital innovative solutions can enrich the educational experience in the K-12 environment. We’ll share how these ...
Content provided by Pearson
Assessment Opinion What Federally Mandated State Tests Are Good For (And What They Aren’t)
Spring 2021 testing is happening. That can be a good thing—if the goal is about more than school accountability.
Stuart Kahl
5 min read
Two people analyze test data
Visual Generation/iStock/Getty
Assessment Opinion The National Assessment Governing Board’s Troubling Gag Order
NAGB's recently released restrictions on how its board members can communicate set a troubling precedent.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
How Are Kids Really Performing This Year?
Has the academic impact of COVID-19 narrowed since last fall? How much progress have students made at each grade level? Get the answers i...
Content provided by Renaissance Learning