Assessment

NAEP Board Wants To Reduce Background Queries

By Lynn Olson — May 14, 2003 7 min read
  • Save to favorites
  • Print

The board that oversees the National Assessment of Educational Progress wants to reduce the number of questions asked of students and teachers that accompany the federally sponsored exams.

A draft of “Background Information Framework for the National Assessment of Educational Progress” is available from the National Assessment Governing Board. (Requires Microsoft Word.)

But at a meeting here this month, researchers and others urged caution in revising what many view as a key feature of the “nation’s report card.”

In addition to monitoring student achievement, NAEP gathers information on students’ experiences in and out of school through hundreds of background questions posed to students, teachers, and administrators at participating sites.

The National Assessment Governing Board, which sets policy for NAEP, held the May 1 meeting to gather comments on a proposed framework for rethinking such questions. The board hopes to focus them more squarely on information needed to report test results or that has a proven relation to student achievement.

While generally supportive of the board’s efforts, respondents recommended prudence in revamping a closely watched part of NAEP’s data-gathering efforts.

The in-school and out-of-school information collected by NAEP is “a crucial tool” for fostering intelligent discourse about education policy and practice, according to Sandra Feldman, the president of the American Federation of Teachers. “Test scores surely tell us something,” she said in written testimony. “But without context, there is little hope of analysis.”

Need for Improvement

Jamel Abedi, a professor of education at the University of California, Los Angeles, said that any changes would “surely have significant impact on the analyses and reporting of NAEP,” including the ability to compare results across time.

Agreement is widespread, however, that the questionnaires, which have been part of NAEP since its inception 34 years ago, need to be improved. The number of background questions has grown considerably over the years, and the governing board, known as NAGB, is concerned both about their quality and the burden on respondents, particularly given the increased frequency Congress has mandated for the assessment.

Some parents and policymakers also contend that a number of those questions are an invasion of privacy and focus too much on students’ beliefs and attitudes.

The “No Child Left Behind” Act of 2001 requires states to take part in NAEP tests in reading and mathematics in grades 4 and 8 every other year. The law, a reauthorization of the Elementary and Secondary Education Act, gives the governing board new authority to approve the background questions. Previously, the board simply advised the Department of Education.

NAGB has proposed refocusing the questions on the assessment’s primary mission: providing a fair and accurate measure of student achievement. Information gathered on students’ race, ethnicity, and socioeconomic status, for instance, helps officials report performance for different student populations.

The board also has proposed maximizing the use of data from other sources in order to limit the amount of background data that NAEP must collect. And it has proposed cycling in some background questions periodically, preferably for limited samples of students, rather than asking the same questions every time.

Many of those who testified agreed with the board’s general principles.

Sally Kilgore, a former director of the office of research in the Education Department’s office of educational research and improvement, advised the board to invest in a “few metrics” of high quality. In particular, she applauded NAGB’s proposal to formulate a better measure of students’ socioeconomic status than their eligibility for subsidized lunches, perhaps by devising an index based on multiple indicators.

“This could be a substantial contribution to this country,” Ms. Kilgore argued.

But Paul W. Holland, a measurement expert with the Princeton, N.J.-based Educational Testing Service, contended that “a huge effort will be put into getting nowhere with that.”

He recommended that the board focus, instead, on collecting better measures of educational practices. “NAEP can give a better description of what is happening in schools to particular groups of students than any other vehicle,” he said, arguing that those portraits are its greatest contribution to education policy.

Others expressed worries that the board would go too far in its quest to reduce the amount of contextual information NAEP collects, either jettisoning some information entirely or collecting it on too infrequent a basis.

“Rather than having fewer noncognitive items, NAEP should have more,” said Harold Wenglinsky, an associate professor at the City University of New York’s Baruch College of Public Affairs. He noted that secondary analyses of NAEP have had a substantial influence on educational research.

Patte Barth, a senior associate with the Washington-based Education Trust, said some contextual questions are so important for students’ opportunity to learn that they should be monitored every two years.

She cited information on teacher credentials; student coursetaking; subject-specific resources, such as access to science labs and libraries; and teachers’ instructional practices. Because such information is central to understanding the achievement gaps between students of different backgrounds, she added, “any background question worth asking is worth reporting by group.”

While some people at the NAGB session questioned the quality of teachers’ self-reports, Ms. Barth and others said ways were available to improve measures of educational practices. “We would not like to see these questions eliminated,” she said, though acknowledging that the number could be streamlined.

Out-of-School Factors

In making decisions about which information to collect, Ms. Barth counseled the board to focus on factors “that are within the school’s control, since these can more easily be changed by public policy.”

But others pressed NAGB to retain information about out-of-school factors that may have a strong relationship to student achievement.

“Given that the research has consistently shown that out-of-school factors explain the lion’s share of variance in achievement, and given, too, that many of these factors, like in-school ones, are neither fixed nor immutable, they should be an important priority for improved background-data collection,” said Ms. Feldman of the AFT.

In one of their rare instances of agreement, both the AFT and the Washington-based Heritage Foundation, a conservative think tank, suggested that the board reconsider producing a parent questionnaire.

Patrick F. Fagan, a fellow at the foundation, also urged the board to collect more information on family structure and on students’ participation in religious communities because of what he says is the critical import of such factors to children’s sense of belonging.

“Perhaps you’re aware of the law, which prohibits NAEP from getting into intrusive questions of family beliefs and attitudes?” John H. Stevens, the NAGB member who is spearheading development of the framework, countered. “How could we possibly ask this in a way that could be viewed by ordinary citizens, as well as members of Congress, as nonintrusive?”

The No Child Left Behind Act prohibits NAEP from asking about “personal or family beliefs and attitudes.” It also requires all questions to be “secular, neutral, and nonideological.”

Mr. Stevens, who is the executive director of the Texas Business and Education Coalition, asked why NAEP, in particular, would be the appropriate or best vehicle for collecting such information.

In an answer that reflects the growing pressures on NAEP, Mr. Fagan responded: “Because it is at the center of the education policy discussion.”

After the meeting, Mr. Stevens pointed out that with limited resources, the governing board has to weigh tradeoffs. “Do we expand the background questions,” he asked, “or maybe grow the samples” to provide more accurate reporting for subgroups of students?

“We need to really be sure that we get the essential part of our mission as right as we can.”

“Expansion is fine,” agreed Thomas H. Fisher, another board member and the director of assessment services for the Florida education department, “as long as Congress and the president enact the necessary changes and provide the necessary funding.”

Related Tags:

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Understanding 'Through-Year' Assessment: What Everyone Should Know
This is a once-in-a-generation opportunity to reconsider our assessment systems. Discover a fresh approach with Through-Year Assessment.
Content provided by New Meridian
Assessment Opinion To Replace Skill Mastery for Seat Time, There Are 3 Requirements
Time for learning and student support take on a whole new meaning in the mastery-based learning model.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Assessment More States Could Drop Their High School Exit Exams
There's movement afoot in nearly half the states that still mandate high school exit exams to end the requirement.
4 min read
A student looks at questions during a college test preparation class at Holton Arms School in Bethesda, Md., on Jan. 17, 2016. The SAT exam will move from paper and pencil to a digital format, administrators announced Tuesday, Jan. 25, 2022, saying the shift will boost its relevancy as more colleges make standardized tests optional for admission.
A student looks at questions during a college test preparation class at Holton Arms School in Bethesda, Md., on Jan. 17, 2016. More states are looking to abandon high school exit exams as support for standardized testing cools.
Alex Brandon/AP
Assessment Cardona Says Standardized Tests Haven't Always Met the Mark, Offers New Flexibility
The U.S. Department of Education is seeking to reinvigorate a little-used pilot program to create new types of assessments.
7 min read
Education Secretary Miguel Cardona speaks during an interview with The Associated Press in his office at the Department of Education on Sept. 20, 2023 in Washington.
Education Secretary Miguel Cardona speaks during an interview with The Associated Press in his office at the Department of Education on Sept. 20, 2023 in Washington.
Mark Schiefelbein/AP