NAEP Board Begins Preparing For Changes
Big changes are looming for the test used to gauge the educational health of the nation's schoolchildren. As one observer warned at a meeting here this month, the next few years could be the "riskiest and most dangerous time" in the history of the nearly 30-year-old National Assessment of Educational Progress.
The board that oversees the federally sponsored assessment began grappling with how to prepare for those changes at its May 10-12 meeting. The preparations include plans for a massive field test of new assessment items and procedures that would have to be administered in 2002 to satisfy legislation Congress is now considering.
Events are unfolding so fast that the National Assessment Governing Board, which usually meets quarterly, has scheduled a special meeting on June 28 in Houston, only the second time that has happened in the governing body's existence.
"This is an extraordinary time for the national assessment," said Gary W. Phillips, the acting commissioner of the National Center for Education Statistics, the branch of the Department of Education that oversees the day-to-day operation of the testing program. "We're going to change things fast."
"We're being loved to death," added Mark D. Musick, the chairman of the governing board, the congressionally authorized body that oversees policy for NAEP.
The national assessment is now taken by national and state samples of students in a range of academic subjects. Generally, each subject is tested once every four years. State participation is voluntary, and the results do not carry any real consequences for states beyond the public reporting of data.
Under President Bush's "No Child Left Behind" plan, however, NAEP would test representative samples of 4th and 8th graders in reading and mathematics in every state every year. States could receive federal financial rewards or be penalized for their progress on state tests, but only if those trends were "confirmed" by NAEP. The president has proposed increasing the assessment's current $40 million budget by an additional $69 million in fiscal 2002 to finance the expansion.
Legislation now pending before Congress to reauthorize the Elementary and Secondary Education Act, the main federal law for K-12 education, would secure Mr. Bush's plan for annual state NAEP tests in both math and reading. The Senate bill, S 1, would require all states to participate in NAEP and would use the results to confirm state test scores, as Mr. Bush has proposed. The House bill, HR 1, would give states the option of using NAEP or another national test, such as those produced by commercial test publishers.
At this month's meeting of NAGB, as the governing board is known, board members expressed concern that the latter option would encourage states to choose whatever measure would make them look best.
"We'll have Ocean Wobegon, from the Atlantic to the Pacific," said Mr. Musick, in a play on Garrison Keillor's fictional Lake Wobegon, whose children are all "above average."
Mr. Musick, the president of the Southern Regional Educational Board, added that with so much testing going on, states could become less willing to participate in NAEP, threatening the future of the assessment.
Secretary of Education Rod Paige, who spoke to the board here May 11, told the NAGB members he shared their concerns arising from the House bill.
"NAEP must be a part of the snapshot that this department uses," he said. "We are taking every action we can to ensure the way you see it is the way it comes out. We think multiple-assessment snapshots would just be a quagmire."
'Praying' for the Future
"I'm going to pray for NAGB because it's going to be a trying time," said William T. Randall, a former state superintendent of education in Colorado, who served as chairman of the governing board from 1994 to 1997.
"The current spotlight on NAEP, I think, is going to raise some real issues."
Mr. Randall was one of three former NAGB chairmen who spoke to the board during the Annapolis meeting. In particular, he cautioned, if the federal government uses NAEP to confirm the gains made on state tests, then the board will have to devise some way to show the progress of students who score "below basic" on the national assessment.
A big proportion of students typically fall into that group. In 1998, for example, 38 percent of 4th graders nationally scored below basic in reading. The governing board currently reports students' scores at four achievement levels: "below basic," "basic," "proficient," and "advanced."
Richard A. Boyd, a former state schools chief in Mississippi, who chaired the board from 1990 to 1992, recalled the initial resistance that many Southern superintendents had to the state-level NAEP because of concerns that it would make their states look bad. The same fears are likely to arise now if NAEP is used to confirm progress—or lack of progress—on state tests, he said.
"There really needs to be a lot of hand-holding with the states out there about exactly what it is you're doing," he warned.
But Chester E. Finn Jr., the chairman of NAGB from 1988 to 1990, noted that it's the U.S. secretary of education who would make the final decision about which states were rewarded or punished under the pending legislation.
"You are the cardiogram, but you're not the cardiologist," said Mr. Finn, who served as the Education Department's assistant secretary for research during the Reagan administration.
The most realistic scenario, he added, is that "for most states, most of the time, there will be no statistically significant changes" on NAEP because the test scores do not change greatly from year to year. "So the likeliest scenario is NAEP will neither confirm nor deny," Mr. Finn said. "It will likely be flat, and states will be judged on their own state tests."
Still, he advised, the increasingly high stakes attached to NAEP make it essential that it be severed from the federal Department of Education.
"The lack of fully independent governance for this sensitive program is particularly troubling as elected officials propose to use NAEP as a 'high stakes' test," he argued. Mr. Finn recommended that NAGB be authorized to manage and direct all NAEP activities.
He also proposed that the federal government compensate schools that agree to participate in the assessment, in addition to paying fully for state participation, as Congress has proposed. In 1999-2000, for example, 48 states signed up for the state- level NAEP, but only 40 actually administered the test because the others could not persuade enough schools to take part.
Districts—or groups of districts or schools—also should be able to participate in the state-level NAEP and receive information about their performance, Mr. Finn said, perhaps with some federal aid.
A proposal by the Council of the Great City Schools to test enough students in some urban districts to yield comparable results across cities was not included in President Bush's budget proposal for fiscal 2002 and is on hold.
Mr. Finn, who described this as the "riskiest and most dangerous time" in the history of the assessment, said that many of the issues he raised would have to be addressed as part of the ongoing evolution of NAEP, "even if there weren't any ESEA reauthorization going on."
A Proposed Field Test
During the meeting, governing-board members were briefed on the outlines of a proposed 2002 field test that gives some sense of the enormous changes in store for NAEP known as the "nation's report card"—if Congress passes the proposed overhaul of the ESEA. They include:
- Combining the national and state samples for NAEP. The "main" and "state" NAEP use the same questions, based on curriculum frameworks adopted since 1990, but they test different samples of students. The field test would combine the administration and sampling procedures for both tests, by having all children tested by naep field-staff members, with the entire cost borne by the federal government. That would enable the same child's test to be counted for both samples, greatly reducing the cost and time burden on states, which now help pay to have classroom teachers administer the state-level NAEP.
- Using more uniform testing times across subjects. At present, NAEP tests in different subjects are administered separately, in part, because the tests are divided into time blocks that vary by grade and subject tested. In the future, it's proposed that all subjects be administered using standard, 25-minute blocks, so that tests in different subjects could be given at the same time, in the same classroom. That change also would be piloted in the 2002 field test.
- Including the scores of students with disabilities or limited English proficiency in the overall test results. Historically, NAEP has not reported scores for tests taken under "nonstandard conditions" by students with disabilities or limited English skills. They were excluded, in part, because of concerns that giving students more time or other accommodations would result in scores that cannot be compared.
Since the late 1990s, that practice has slowly begun to change as new tests have been created. For the 2002 reading test, NAEP plans to include students with accommodations as part of its state and national samples and include their scores in reporting the results. The assessment program will not, however, permit accommodations that could change the reading skills being measured, such as reading aloud to children, or permitting students with limited English skills to use dictionaries or translations of test items.
- Permitting students to identify themselves as members of more than one race or ethnic group. For the first time, in 2003, students who take the national assessment will be able to identify themselves as members of more than one racial or ethnic group, in keeping with new federal reporting requirements first used by the 2000 Census. The new classifications could change the outcomes for particular subgroups, and affect the reporting of achievement gaps, by including more or fewer students in specific racial and ethnic categories than in the past. The new classification categories also would be piloted in the 2002 field test.
- Adding new test items in reading and math. The field test would have to include a large enough pool of new test items in reading and math to begin testing in those subjects annually, starting in 2003. NAEP will have access to about 1,600 NAEP-like math items and 1,800 NAEP-like reading items written for the voluntary new national tests proposed under the Clinton administration, but additional test items would be needed.
Board members meeting in Annapolis expressed concerns about whether they would have the money to begin preparing for the field test this year. Roy Truby, the board's executive director, estimated that about $5 million would be needed this fiscal year, in addition to the $40 million already appropriated for NAEP's ongoing work.
Secretary Paige assured board members that he was aware of the concern and of the need to get started immediately.
Many of the changes now being contemplated as a result of President Bush's accountability- driven "No Child Left Behind" plan are in line with a proposed redesign of NAEP that the governing board has been working on for several years.
But some of the changes also are controversial. For example, some people are worried that including students who receive accommodations in NAEP results could lower test scores, particularly in states with high populations of special-needs students, and disrupt trend lines that allow states to track results over time.
Mr. Truby predicted that including the scores of students with accommodations would cause test scores to dip in the short term, but that it would not be a major change.
Federal law now requires states to test all their students, to the extent practicable, added Edward H. Haertel, a NAGB member and an education professor at Stanford University. "NAEP is going to have to keep up with the times."
Vol. 20, Issue 37, Pages 25,28