Experts Urge Caution in Expanding State-Level NAEP

By Robert Rothman — March 11, 1992 5 min read

The report states that lifting the ban could stifle states’ and districts innovations in assessment, and could destroy NAEP’s value as an independent monitor of student achievement.

A group of leading scholars convened to evaluate the first state-level assessment of student achievement last week urged caution in continuing the program.

In a report issued here, the panel of the National Academy of Education said the assessment of 8th graders in mathematics conducted in 1990 by the National Assessment of Educational Progress demonstrated that such a test is “technically feasible” and generated useful information.

But the panel urged the Congress to continue the state-level assessment in 1994 on a trial basis only, and recommended against lifting the prohibition barring school districts and schools from reporting NAEP scores.

The report also calls on the National Assessment Governing Board not to abandon the traditional method of reporting NAEP results as it pursues its controversial practice of reporting performance against standards for student achievement.

“The Congress had good sense in calling for a trial assessment,” said Robert Glaser, the director of the Learning Research and Development Center at the University of Pittsburgh and the co-chairman of the academy panel. “An endeavor of this kind should undergo clinical trials before something becomes an established part of the education scene.”

Members of the N.A.G.B., meeting here last week, said they supported most of the panel’s recommendations. But they disagreed strongly with the proposal not to allow districts and schools to use NAEP data.

Richard A. Boyd, the chairman of the N.A.G.B., also argued that the academy panel’s proposal for two methods of reporting NAEP data would be confusing and redundant.

“It’s like saying I should wear suspenders and a belt,’' Mr. Boyd said. “Because the belt may break, I ought to have suspenders on, too.”

Including the Excluded

A Congressionally mandated project that has for 20 years tested national samples of students, NAEP was authorized by the Congress in 1988 to expand to permit the first-ever state-by-state comparisons of student achievement.

In addition to the 1990 math assessment, the results of which were issued last June, the Congress also authorized state-level tests in 1992 of 4th graders in reading and of 4th and 8th graders in math.

To carry out a Congressional mandate to provide an independent evaluation of the state-level assessments, the National Center for Education Statistics, which oversees NAEP, commissioned the 18-member academy panel.

In a preliminary report issued last year, the panel found that the 1990 state math assessment had worked well and recommended that the results be released as scheduled. But it argued that the Congress should only expand the state-level assessment on a trial basis, to include testing at grades 4, 8, and 12 in three subjects. (See Education Week, April 17, 1991.)

The new report, the first of three planned by the academy panel, confirms the earlier finding that the 1990 state-level math assessment was conducted well.

It notes, however, that the proportion of students who were permitted by law to be excluded from the tests--those who attend private schools or who have a handicapping condition or severely limited English proficiency--varied widely from state to state. This variation could have affected the states’ overall rankings, the report suggests.

To improve the information on states’ achievement, the panel recommended that private-school students be included in the state samples and that the N.C.E.S. study state policies for excluding handicapped students. It also urged a study of the feasibility of conducting an assessment in Spanish.

The panel also urged that, beginning in 1994, a sample of out-of-school 17-year-olds be assessed. Because such an assessment would be costly, however, the panel recommended that it take place at the national level only.

Critical of Standards

In examining the content of the 1990 assessment, the panel found that it reflected “a reasonable compromise between current instructional practice and the new views of mathematical competence being articulated by professional organizations.”

But it also maintained that the frameworks for the assessment should remain “evolutionary,” and should over time include items that were more difficult, more diverse, and more numerous over all, to reflect the vision of reformers.

The panel’s report also suggests that, in light of the considerable amount of time needed to develop new assessments, the Congress authorize future NAEP exams several years in advance.

“One-year-at-a-time reauthorizations and extensions just prior to the scheduled field tests are not adequate,” the report states.

In discussing the way the results were reported, the panel concurred with several panels of experts who had criticized the governing board’s procedure for setting standards for the assessment. Under that system, the board reported the proportion of students who performed at the basic, proficient, and advanced levels on the assessment.

In addition to raising the technical concerns, the panel also contended that “the utility of reporting what students should know compared to what they do know has yet to be demonstrated.”

A Valuable Monitor

In looking to the future of the state-level assessments, the panel reiterated its earlier conclusion that the program be continued on a trial basis only.

And, after reviewing commissioned papers arguing for and against lifting the prohibition against district- and school-level use of the NAEP data, the panel concluded that the reasons to oppose such a move are “more compelling.”

One panel member--Gordon M. Ambach, the executive director of the Council of Chief State School Officers--dissented on the issue, however.

The report states that lifting the ban could stifle states’ and districts innovations in assessment, and could destroy NAEP’s value as an independent monitor of student achievement.

“NAEP has played and should play the same role for education that economic indicators have played in signaling changes in the status of the economy,” the report states.

A version of this article appeared in the March 11, 1992 edition of Education Week as Experts Urge Caution in Expanding State-Level NAEP