NAEP Board Wants To Reduce Background Queries

By Lynn Olson — May 14, 2003 7 min read
  • Save to favorites
  • Print

The board that oversees the National Assessment of Educational Progress wants to reduce the number of questions asked of students and teachers that accompany the federally sponsored exams.

A draft of “Background Information Framework for the National Assessment of Educational Progress” is available from the National Assessment Governing Board. (Requires Microsoft Word.)

But at a meeting here this month, researchers and others urged caution in revising what many view as a key feature of the “nation’s report card.”

In addition to monitoring student achievement, NAEP gathers information on students’ experiences in and out of school through hundreds of background questions posed to students, teachers, and administrators at participating sites.

The National Assessment Governing Board, which sets policy for NAEP, held the May 1 meeting to gather comments on a proposed framework for rethinking such questions. The board hopes to focus them more squarely on information needed to report test results or that has a proven relation to student achievement.

While generally supportive of the board’s efforts, respondents recommended prudence in revamping a closely watched part of NAEP’s data-gathering efforts.

The in-school and out-of-school information collected by NAEP is “a crucial tool” for fostering intelligent discourse about education policy and practice, according to Sandra Feldman, the president of the American Federation of Teachers. “Test scores surely tell us something,” she said in written testimony. “But without context, there is little hope of analysis.”

Need for Improvement

Jamel Abedi, a professor of education at the University of California, Los Angeles, said that any changes would “surely have significant impact on the analyses and reporting of NAEP,” including the ability to compare results across time.

Agreement is widespread, however, that the questionnaires, which have been part of NAEP since its inception 34 years ago, need to be improved. The number of background questions has grown considerably over the years, and the governing board, known as NAGB, is concerned both about their quality and the burden on respondents, particularly given the increased frequency Congress has mandated for the assessment.

Some parents and policymakers also contend that a number of those questions are an invasion of privacy and focus too much on students’ beliefs and attitudes.

The “No Child Left Behind” Act of 2001 requires states to take part in NAEP tests in reading and mathematics in grades 4 and 8 every other year. The law, a reauthorization of the Elementary and Secondary Education Act, gives the governing board new authority to approve the background questions. Previously, the board simply advised the Department of Education.

NAGB has proposed refocusing the questions on the assessment’s primary mission: providing a fair and accurate measure of student achievement. Information gathered on students’ race, ethnicity, and socioeconomic status, for instance, helps officials report performance for different student populations.

The board also has proposed maximizing the use of data from other sources in order to limit the amount of background data that NAEP must collect. And it has proposed cycling in some background questions periodically, preferably for limited samples of students, rather than asking the same questions every time.

Many of those who testified agreed with the board’s general principles.

Sally Kilgore, a former director of the office of research in the Education Department’s office of educational research and improvement, advised the board to invest in a “few metrics” of high quality. In particular, she applauded NAGB’s proposal to formulate a better measure of students’ socioeconomic status than their eligibility for subsidized lunches, perhaps by devising an index based on multiple indicators.

“This could be a substantial contribution to this country,” Ms. Kilgore argued.

But Paul W. Holland, a measurement expert with the Princeton, N.J.-based Educational Testing Service, contended that “a huge effort will be put into getting nowhere with that.”

He recommended that the board focus, instead, on collecting better measures of educational practices. “NAEP can give a better description of what is happening in schools to particular groups of students than any other vehicle,” he said, arguing that those portraits are its greatest contribution to education policy.

Others expressed worries that the board would go too far in its quest to reduce the amount of contextual information NAEP collects, either jettisoning some information entirely or collecting it on too infrequent a basis.

“Rather than having fewer noncognitive items, NAEP should have more,” said Harold Wenglinsky, an associate professor at the City University of New York’s Baruch College of Public Affairs. He noted that secondary analyses of NAEP have had a substantial influence on educational research.

Patte Barth, a senior associate with the Washington-based Education Trust, said some contextual questions are so important for students’ opportunity to learn that they should be monitored every two years.

She cited information on teacher credentials; student coursetaking; subject-specific resources, such as access to science labs and libraries; and teachers’ instructional practices. Because such information is central to understanding the achievement gaps between students of different backgrounds, she added, “any background question worth asking is worth reporting by group.”

While some people at the NAGB session questioned the quality of teachers’ self-reports, Ms. Barth and others said ways were available to improve measures of educational practices. “We would not like to see these questions eliminated,” she said, though acknowledging that the number could be streamlined.

Out-of-School Factors

In making decisions about which information to collect, Ms. Barth counseled the board to focus on factors “that are within the school’s control, since these can more easily be changed by public policy.”

But others pressed NAGB to retain information about out-of-school factors that may have a strong relationship to student achievement.

“Given that the research has consistently shown that out-of-school factors explain the lion’s share of variance in achievement, and given, too, that many of these factors, like in-school ones, are neither fixed nor immutable, they should be an important priority for improved background-data collection,” said Ms. Feldman of the AFT.

In one of their rare instances of agreement, both the AFT and the Washington-based Heritage Foundation, a conservative think tank, suggested that the board reconsider producing a parent questionnaire.

Patrick F. Fagan, a fellow at the foundation, also urged the board to collect more information on family structure and on students’ participation in religious communities because of what he says is the critical import of such factors to children’s sense of belonging.

“Perhaps you’re aware of the law, which prohibits NAEP from getting into intrusive questions of family beliefs and attitudes?” John H. Stevens, the NAGB member who is spearheading development of the framework, countered. “How could we possibly ask this in a way that could be viewed by ordinary citizens, as well as members of Congress, as nonintrusive?”

The No Child Left Behind Act prohibits NAEP from asking about “personal or family beliefs and attitudes.” It also requires all questions to be “secular, neutral, and nonideological.”

Mr. Stevens, who is the executive director of the Texas Business and Education Coalition, asked why NAEP, in particular, would be the appropriate or best vehicle for collecting such information.

In an answer that reflects the growing pressures on NAEP, Mr. Fagan responded: “Because it is at the center of the education policy discussion.”

After the meeting, Mr. Stevens pointed out that with limited resources, the governing board has to weigh tradeoffs. “Do we expand the background questions,” he asked, “or maybe grow the samples” to provide more accurate reporting for subgroups of students?

“We need to really be sure that we get the essential part of our mission as right as we can.”

“Expansion is fine,” agreed Thomas H. Fisher, another board member and the director of assessment services for the Florida education department, “as long as Congress and the president enact the necessary changes and provide the necessary funding.”

Related Tags:


Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Accelerate Learning with Project-Based Learning
Earlier this year, the George Lucas Educational Foundation released four new studies highlighting how project-based learning (PBL) helps accelerate student learning—across age groups, multiple disciplines, and different socio-economic statuses. With this year’s emphasis on unfinished
Content provided by SmartLab Learning
School & District Management Live Online Discussion Principal Overload: How to Manage Anxiety, Stress, and Tough Decisions
According to recent surveys, more than 40 percent of principals are considering leaving their jobs. With the pandemic, running a school building has become even more complicated, and principals' workloads continue to grow. If we

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP
Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."