NAEP Board Wants To Reduce Background Queries

By Lynn Olson — May 14, 2003 7 min read
  • Save to favorites
  • Print

The board that oversees the National Assessment of Educational Progress wants to reduce the number of questions asked of students and teachers that accompany the federally sponsored exams.

A draft of “Background Information Framework for the National Assessment of Educational Progress” is available from the National Assessment Governing Board. (Requires Microsoft Word.)

But at a meeting here this month, researchers and others urged caution in revising what many view as a key feature of the “nation’s report card.”

In addition to monitoring student achievement, NAEP gathers information on students’ experiences in and out of school through hundreds of background questions posed to students, teachers, and administrators at participating sites.

The National Assessment Governing Board, which sets policy for NAEP, held the May 1 meeting to gather comments on a proposed framework for rethinking such questions. The board hopes to focus them more squarely on information needed to report test results or that has a proven relation to student achievement.

While generally supportive of the board’s efforts, respondents recommended prudence in revamping a closely watched part of NAEP’s data-gathering efforts.

The in-school and out-of-school information collected by NAEP is “a crucial tool” for fostering intelligent discourse about education policy and practice, according to Sandra Feldman, the president of the American Federation of Teachers. “Test scores surely tell us something,” she said in written testimony. “But without context, there is little hope of analysis.”

Need for Improvement

Jamel Abedi, a professor of education at the University of California, Los Angeles, said that any changes would “surely have significant impact on the analyses and reporting of NAEP,” including the ability to compare results across time.

Agreement is widespread, however, that the questionnaires, which have been part of NAEP since its inception 34 years ago, need to be improved. The number of background questions has grown considerably over the years, and the governing board, known as NAGB, is concerned both about their quality and the burden on respondents, particularly given the increased frequency Congress has mandated for the assessment.

Some parents and policymakers also contend that a number of those questions are an invasion of privacy and focus too much on students’ beliefs and attitudes.

The “No Child Left Behind” Act of 2001 requires states to take part in NAEP tests in reading and mathematics in grades 4 and 8 every other year. The law, a reauthorization of the Elementary and Secondary Education Act, gives the governing board new authority to approve the background questions. Previously, the board simply advised the Department of Education.

NAGB has proposed refocusing the questions on the assessment’s primary mission: providing a fair and accurate measure of student achievement. Information gathered on students’ race, ethnicity, and socioeconomic status, for instance, helps officials report performance for different student populations.

The board also has proposed maximizing the use of data from other sources in order to limit the amount of background data that NAEP must collect. And it has proposed cycling in some background questions periodically, preferably for limited samples of students, rather than asking the same questions every time.

Many of those who testified agreed with the board’s general principles.

Sally Kilgore, a former director of the office of research in the Education Department’s office of educational research and improvement, advised the board to invest in a “few metrics” of high quality. In particular, she applauded NAGB’s proposal to formulate a better measure of students’ socioeconomic status than their eligibility for subsidized lunches, perhaps by devising an index based on multiple indicators.

“This could be a substantial contribution to this country,” Ms. Kilgore argued.

But Paul W. Holland, a measurement expert with the Princeton, N.J.-based Educational Testing Service, contended that “a huge effort will be put into getting nowhere with that.”

He recommended that the board focus, instead, on collecting better measures of educational practices. “NAEP can give a better description of what is happening in schools to particular groups of students than any other vehicle,” he said, arguing that those portraits are its greatest contribution to education policy.

Others expressed worries that the board would go too far in its quest to reduce the amount of contextual information NAEP collects, either jettisoning some information entirely or collecting it on too infrequent a basis.

“Rather than having fewer noncognitive items, NAEP should have more,” said Harold Wenglinsky, an associate professor at the City University of New York’s Baruch College of Public Affairs. He noted that secondary analyses of NAEP have had a substantial influence on educational research.

Patte Barth, a senior associate with the Washington-based Education Trust, said some contextual questions are so important for students’ opportunity to learn that they should be monitored every two years.

She cited information on teacher credentials; student coursetaking; subject-specific resources, such as access to science labs and libraries; and teachers’ instructional practices. Because such information is central to understanding the achievement gaps between students of different backgrounds, she added, “any background question worth asking is worth reporting by group.”

While some people at the NAGB session questioned the quality of teachers’ self-reports, Ms. Barth and others said ways were available to improve measures of educational practices. “We would not like to see these questions eliminated,” she said, though acknowledging that the number could be streamlined.

Out-of-School Factors

In making decisions about which information to collect, Ms. Barth counseled the board to focus on factors “that are within the school’s control, since these can more easily be changed by public policy.”

But others pressed NAGB to retain information about out-of-school factors that may have a strong relationship to student achievement.

“Given that the research has consistently shown that out-of-school factors explain the lion’s share of variance in achievement, and given, too, that many of these factors, like in-school ones, are neither fixed nor immutable, they should be an important priority for improved background-data collection,” said Ms. Feldman of the AFT.

In one of their rare instances of agreement, both the AFT and the Washington-based Heritage Foundation, a conservative think tank, suggested that the board reconsider producing a parent questionnaire.

Patrick F. Fagan, a fellow at the foundation, also urged the board to collect more information on family structure and on students’ participation in religious communities because of what he says is the critical import of such factors to children’s sense of belonging.

“Perhaps you’re aware of the law, which prohibits NAEP from getting into intrusive questions of family beliefs and attitudes?” John H. Stevens, the NAGB member who is spearheading development of the framework, countered. “How could we possibly ask this in a way that could be viewed by ordinary citizens, as well as members of Congress, as nonintrusive?”

The No Child Left Behind Act prohibits NAEP from asking about “personal or family beliefs and attitudes.” It also requires all questions to be “secular, neutral, and nonideological.”

Mr. Stevens, who is the executive director of the Texas Business and Education Coalition, asked why NAEP, in particular, would be the appropriate or best vehicle for collecting such information.

In an answer that reflects the growing pressures on NAEP, Mr. Fagan responded: “Because it is at the center of the education policy discussion.”

After the meeting, Mr. Stevens pointed out that with limited resources, the governing board has to weigh tradeoffs. “Do we expand the background questions,” he asked, “or maybe grow the samples” to provide more accurate reporting for subgroups of students?

“We need to really be sure that we get the essential part of our mission as right as we can.”

“Expansion is fine,” agreed Thomas H. Fisher, another board member and the director of assessment services for the Florida education department, “as long as Congress and the president enact the necessary changes and provide the necessary funding.”

Related Tags:


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Professional Development Webinar
Disrupting PD Day in Schools with Continuous Professional Learning Experiences
Hear how this NC School District achieved district-wide change by shifting from traditional PD days to year-long professional learning cycles
Content provided by BetterLesson
Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
Teacher Perspectives: What is the Future of Virtual Education?
Hear from practicing educators on how virtual and hybrid options offer more flexibility and best practices for administrative support.
Content provided by Class

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment A Huge Publisher and a Big Testing Company Are Teaming Up. What This Means for Educators
Four key questions to consider about how the pairing of Houghton Mifflin Harcourt and NWEA might affect educators.
3 min read
Students testing.
Assessment Spotlight Spotlight on Assessment
This Spotlight will help you examine updated testing guidance from the U.S. Dept. of Ed, analyze college-placement test scores, and more.

Assessment Opinion We Are More Than Our Grades: A Student's Perspective
Students have come to believe that their GPA and test scores are the ultimate reflection of their self-worth, writes a college senior.
Bailey Striepling
3 min read
Conceptual illustration of Students emerging from a field of giant discarded letter grades.
Vanessa Solis/Education Week and iStock/Getty Images
Assessment Letter to the Editor Grading for Growth Through Competency-Based Education
Competency-based education can better prepare today's children for tomorrow's challenges, writes this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.