A group of leading scholars formed to evaluate the National Assessment of Educational Progress’s trial state-level assessments has concluded that the program has essentially “worked’’ and should be conducted every year.
However, in a report released last week, the panel from the National Academy of Education also went on to urge that the federal government keep monitoring the assessments and recommended against a Clinton Administration proposal to lift a prohibition barring the reporting of scores at the school and school-district levels.
“Our number-one question was whether a continuing state NAEP should be authorized and we said, ‘Yes, state NAEP has been shown to be technically feasible and has provided data that is useful,’'' Robert Linn, who is a co-chairman of the panel and an education professor at the University of Colorado at Boulder, said in an interview.
“The recommendation for continuing evaluation just reflects a desire to take into account the fact that these assessments will change over time,’' he said. “And there are some issues that we said needed continued work.’'
A federal program that has for 20 years tested national samples of students, NAEP underwent a controversial expansion in 1988 to permit on a trial basis the first-ever state-by-state comparisons of student achievement.
Those assessments were conducted in 1990 in mathematics and again in 1992 when 4th graders were tested in reading and 4th and 8th graders were assessed in math. More than 40 states have already agreed to take part in the 1994 assessments.
To carry out a Congressional mandate to provide an independent evaluation of the state-level assessments, the National Center for Education Statistics, which oversees NAEP, commissioned the 18-member academy panel.
Problems Remain
Last week’s 193-page report was the academy’s final evaluation of the 1992 state-level assessments. It comes just as Congress is preparing to take up proposals for reauthorizing the NAEP program.
In an earlier report, the academy recommended continuing the program only on a trial basis.
“Now they’ve been through two rounds, and I think they’re really becoming quite confident this is a good idea and should be continued,’' Commissioner of Education Statistics Emerson J. Elliott said in an interview.
However, the report says that ongoing evaluation of the program is still needed, in part because the program has yet to be fully implemented at three grade levels--4th, 8th, and 12th.
Moreover, the report raises concerns over the rates at which the schools initially targeted to take part in the program are actually participating.
According to NAEP policy, statisticians select those schools based on such factors as the socioeconomic makeup of their students. If those schools decline to participate, the state officials administering the assessments must substitute schools with similar characteristics.
The panel members found, however, that the initial participation rates were increasingly varied among states and that there were more substitutions in 1992 than there were in 1990.
And states that had large numbers of substitutions also had higher scores on the assessments--a finding that suggests NAEP results could be inflated.
“The academy is legitimately saying, ‘Hey, this is something we need to keep an eye on,’ and we would agree,’' said Mark D. Musick, the chairman of the National Assessment Governing Board, which sets policy for NAEP.
But board officials also pointed out that making the requirements for high initial participation too rigorous might also discourage some states from joining. Participation-rate requirements already in place have forced two large states--Illinois and Ohio--to drop out of the 1994 state assessments.
The new report also reiterates strong objections to the procedures used to develop achievement levels for the assessments. NAEP assessments gauge student achievement against three standards--basic, proficient, and advanced--and academy members in a previous report said the methods used to determine those levels were flawed.
Dropping the Ban
The report also weighed in on an issue that is expected to be one of the most controversial as Congress considers the NAEP reauthorization by recommending against dropping the ban on reporting NAEP results below the state level.
In its reauthorization proposal, the Clinton Administration calls for allowing schools and districts for the first time to report their own results--provided that they pay for it and that they follow NAEP procedures.
But the report says that change could stifle states’ and school districts’ innovations in assessment and undermine the program’s value as a thermometer of the nation’s educational health by raising the stakes.
If the ban is lifted anyway, the report says, then it should be done on a trial basis and appropriate statistical and procedural requirements should be set.
Arguing that it would discourage states’ participation, the report also opposes another provision of the Administration bill that would allow the federal government to transfer some program costs to states.
Mr. Elliott said he did not expect the Administration to change its legislative proposal in response to the criticisms.
