Parents will have new powers to withdraw their children from the federal assessment program, and students themselves now will be able to bow out, under a provision that experts warn could undermine the accuracy of test scores.
The surprise, last-minute addition to the new federal K-12 education law requires that parents be informed that their children have been selected to take part in the National Assessment of Educational Progress. If a parent doesn’t want his or her child to be tested, the child will be pulled out of the sample. And students, acting on their own, may pick and choose the questions they answer or even walk out of the test-taking at any time.
Because of the new rule, scorers may struggle to interpret test results, and eventually the usefulness of NAEP could be reduced, testing experts argue.
“The bigger the nonparticipation rate, the more anybody who follows this stuff will scratch their head and say, ‘Do I believe it?’” said Daniel M. Koretz, an education professor at Harvard University. “It’s so unpredictable, and that’s part of what makes it so threatening” to the testing program.
The role of the national assessment was a major sticking point in the “No Child Left Behind” Act of 2001, which President Bush signed into law last month. Mr. Bush had proposed that a state’s NAEP scores be used to validate student-achievement growth on the state’s own tests, and that a state’s NAEP results be one of the factors in determining whether a state received federal rewards or was penalized based on its students’ achievement.
Conservative critics argued that those features would have expanded the testing program’s influence and, by extension, the federal government’s role in state and local school decisions. They urged that a state be allowed to choose another exam—such as a commercially available test— to play the role that Mr. Bush envisioned for NAEP, which tests a sampling of students in key subjects.
In the end, the law requires states to participate in NAEP to continue receiving federal funding, but the assessment’s scores won’t be used in calculating rewards or withholding aid. Many conservatives opposed the bill, in part, because of the testing program’s new role.
Surprise Package
In an attempt to appease those critics, Rep. John A. Boehner, the chairman of the House Education and the Workforce Committee, added the new NAEP rules in the final drafting of the bill, which reauthorized the Elementary and Secondary Education Act. Neither the Senate nor the House versions of the bill included such a provision.
Mr. Boehner, an Ohio Republican, added the clause in the hope of winning the support of those who objected to expanding the influence of the federal testing program, said Heather B. Valentine, the House committee’s press secretary. The chairman also believes it is “prudent policy” to give parents and students a choice on NAEP participation, Ms. Valentine added.
Although some of the biggest detractors of NAEP supported the provision, they did not seek the modifications, according to Jennifer A. Marshall, the director of family studies at the Family Research Council, a Washington-based advocacy group.
Still, giving parents and students the choice of taking the tests may help smooth opposition to the program and should not cause problems for NAEP, Ms. Marshall said.
“I don’t think it will be a problem on a grand scale,” she said. “It’s nothing they can’t fix with an adjustment to their sample elsewhere.”
But testing experts say solving the problems posed by the new requirements won’t be simple.
Because the program relies on relatively small samples of students to generate overall national or statewide scores, at least one prominent testing expert involved in the governance of NAEP cautions that changes in student performance on just a few questions will lead to large fluctuations in test scores.
“Overall performance of the whole system doesn’t change very fast, so anything that disturbs it can show up pretty fast,” said Edward H. Haertel, a professor of education at Stanford University and a member of the National Assessment Governing Board, which oversees the testing program. “On the face of it, it could be problematic,” he said of the opt-out provision.
Another researcher said NAEP’s developers will be able to estimate the impact of the new rules once the assessment is given next year, which will be the first testing under those rules.
They can do some checking and compare it to what they know about previous NAEP administrations to see if it’s a big problem or a little problem,” said Eva L. Baker, a co-director of the National Center for Research on Evaluation, Standards, and Student Testing, a federally financed project at the University of California, Los Angeles.
Issues of Analysis
Under existing NAEP rules, statisticians select schools to produce a sample of students to take portions of the exam. The overall samples are designed to mirror the demographics of particular states or the nation. Test results are then reported as a single score in each state and for the whole country. Students do not receive their individual scores on NAEP tests, and the scores are not used in making academic decisions about individual students or in evaluating school districts.
Since NAEP began collecting scores from individual states in 1990, state participation has been voluntary. NAEP collects state-by-state and national scores in reading, writing, mathematics, and science. It also administers tests in U.S. history, geography, civics, and the arts, and reports the scores on the national level.
Until now, federal law was silent on whether parents and students could opt out of NAEP. Federal regulations never compelled student participation, however, and state and school policies could give parents the option of holding their children out of the testing program.
Still, the explicit opt-out provision in the new law could make nonparticipation more likely.
And the arbitrary nature of parents’ and students’ decisions will confuse testing analysts, in their view, as they try to understand test scores.
Analysts won’t know, for example, if parents pull a child out of the testing because they object to the numbers of tests administered in schools, they fear the child is likely to do poorly, or they are concerned for any number of other reasons.
Likewise, the analysts won’t know if a student refuses to answer a question because he or she didn’t know the answer, objected to its content, or decided to nap instead of taking the test.
Researchers may be able to estimate the impact of nonparticipation on scores, said Mr. Koretz of Harvard, but only if NAEP collects such information as test-takers’ social background, grades, and performance on other tests.
If researchers know a child’s past academic performance, he said, they can estimate how the student would have performed on a particular test question left blank on his or her answer sheet. If all they know is the child’s demographic background, they can compare blank answer sheets with those of similar students to guess how the child otherwise would score on the test. But because of the nature of the new rules, federal officials won’t be able to force students to provide such information.
The new rules also will make it impossible for the National Assessment Governing Board to conduct a study comparing the performance of students under the old rules with a separate group operating under the new ones, said Mr. Haertel, who is the chairman of the governing board’s methodology committee.
Similar studies have helped the board gauge the impact of policy changes on NAEP test results in the past, he said.
At the next meeting of Mr. Haertel’s committee at the end of this month, the panel will begin writing new policies that will inform parents and students of their right to opt out of the national assessment.
