Researchers Helping NAEP Board Find Value in Background Queries

October 02, 2002 4 min read

Researchers weighed in last week on ways to rethink the often-controversial background questions that accompany “the nation’s report card,” in an attempt to enhance the assessment’s research and policymaking value.

They did so at the request of the National Assessment of Educational Progress’ governing board, which has new authority to approve the background questions posed to students, teachers, and school administrators. Before the “No Child Left Behind” Act of 2001, the board simply advised the Department of Education.

The National Assessment Governing Board, which is aiming to devise a framework for approaching the background questions by next May, asked researchers to submit papers on a number of related topics.

Some questions, those dealing with race, ethnicity, and socioeconomic status, have long been the backbone of NAEP. Such information helps education officials monitor achievement for different student populations over time.

But the questions also delve into other matters, such as students’ behavior, school experiences, and home environments, as well as teachers’ backgrounds and instructional practices. Some policymakers and parents contend that those kinds of questions are an invasion of privacy.

Narrowing the Focus

John H. Stevens, a governing-board member appointed by President Bush, pointed out that the background questions have been around since the inception of the 33-year-old testing program, which periodically samples students in a variety of subjects. Mr. Stevens, the executive director of the Texas Business and Education Coalition and the board member who is spearheading the background-question effort, said last week’s meeting represented just the beginning of the process.

The No Child Left Behind Act ratchets up the importance of NAEP. All states will now be required to administer the tests. In addition, NAEP exams will be given at least every other year in reading and mathematics for 4th and 8th graders and be used as an independent benchmark for gauging states’ progress on their own tests.

“NAEP has only realized 10 or 15 percent of its capability,” David W. Grissmer, an education researcher at the Santa Monica, Calif.-based RAND Corp., told governing-board members at their Sept. 24 workshop.

He suggested that improvements to the background questions, coupled with other changes in data collection and sampling procedures, would substantially improve NAEP’s value. Mr. Grissmer estimated that the governing board could eliminate two-thirds of the current background questions without harming research, if the remaining questions were more carefully crafted and clearly focused.

Mr. Grissmer also recommended creating a “simple” parent survey to improve the research value.

Moreover, he argued that some current background questions “encourage misuse.” He cited a question about students’ television-viewing habits that he characterized as being driven by overly simple research hypotheses.

Ina V.S. Mullis, a researcher at Boston College, made the point, echoed by others, that the board should devise a focused set of questions because students and teachers have limited time and patience.

“Think of a primary purpose for these background questions,” she said. “Pick something manageable ... something that at least might be feasible.”

Paul E. Barton, an independent testing expert, called the background questions an “under-attended aspect of NAEP” in his paper.

Mr. Barton said a dramatic expansion in background questions started in 1984, largely to reflect a desire to explain levels of achievement.

Herbert J. Walberg, a professor emeritus at the University of Illinois at Chicago and a visiting fellow at Stanford University, offered a laundry list of potential indicators the questions might address: school choice, class size, and retention of failing students in their current grades, among them.

The national assessment, he acknowledged, could not reasonably examine all the areas, though items could be “rotated” among students to minimize the burden.

No More ‘Ornaments’

One researcher, however, was skeptical of NAEP’s value in shedding light on how schools and teachers affect student achievement.

Michael Podgursky, an economics professor at the University of Missouri, said some responses from students, and even teachers, might not be very reliable. Moreover, no prior data are available on the students tested, he pointed out.

State tests hold far greater promise for answering questions on what factors affect student achievement, Mr. Podgursky argued, because states have increasingly rich data on students, teachers, and classroom practices. Given the limited resources for NAEP, he said, money should not go toward getting at such issues in NAEP.

“I’d love to see a bigger sample size,” he said, rather than “adding more ornaments.”

Under the No Child Left Behind Act, a revision of the Elementary and Secondary Education Act, NAEP must not “evaluate or assess personal or family beliefs and attitudes.” Congress said that provision does not prohibit the use of “nonintrusive” background questions if they have a demonstrated relationship to academic achievement. In addition, all questions must be “secular, neutral, and nonideological,” a provision of the 1994 edition of the ESEA as well.

“People don’t want the national assessment to intrude,” Mr. Stevens of the governing board said.

The board earlier this year approved definitions of secular, neutral, and nonideological and removed a background item related—though vaguely—to religion when it approved the reading and math tests for 2003.

In a set of questions about the amount of time students spend outside of school reading nonfiction, the board this summer axed an item that said “books about religion or philosophy.”

Related Tags:


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Reading & Literacy Webinar
Addressing Learning Loss: What Schools Need to Accelerate Reading Instruction in K-3
When K-3 students return to classrooms this fall, there will be huge gaps in foundational reading skills. Does your school or district need a plan to address learning loss and accelerate student growth? In this
Content provided by PDX Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning
Classroom Technology Webinar Making Big Technology Decisions: Advice for District Leaders, Principals, and Teachers
Educators at all levels make decisions that can have a huge impact on students. That’s especially true when it comes to the use of technology, which was activated like never before to help students learn

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment How Can Teachers Better Understand Students? A New Breed of Assessment Will Try to Help
Researchers will work to create formative assessments that can give teachers a window into students’ emerging identities and strengths
4 min read
In this Nov. 19, 2020, file photo, sixth-grade students listen to instruction in class at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Researchers hope to create new assessments to help teachers gain deeper insights into the identities and strengths of their students, like these 6th graders at Cape Cod Lighthouse Charter School in East Harwich, Mass.
Elise Amendola/AP
Assessment Opinion It's Time We Begin Using Assessments to Look Forward, Instead of Back
Schools do not get much value from high-stakes tests. Many are now allowing schools to use better assessments to guide student learning.
Seth Feldman
5 min read
shutterstock 19525837
Assessment Opinion Grading Has Always Been an Imperfect Exercise. COVID-19 Made It Worse
It’s hard reducing the complexity of each student’s social, emotional, and academic learning to a letter grade. Maybe we’re doing it wrong.
Lory Walker Peroff
4 min read
A student's grades are unknown
Robert Neubecker for Education Week
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
Facing the Future Together: Digital Innovative Solutions
Join us to discuss how digital innovative solutions can enrich the educational experience in the K-12 environment. We’ll share how these ...
Content provided by Pearson