NAEP To Create Three Standards For Performance
Washington--The governing board of the National Assessment of Educational Progress has unanimously approved a plan to set the first national standards for student achievement.
The plan, a modified version of a proposal first considered by the board in December, was adopted here May 11 and is expected to go into effect this fall with the 1990 assessment in mathematics, the first to provide state-level data on performance. If it proves successful, board members said, it will be expanded to all subject areas in 1992.
In contrast to the current practice of simply describing how students performed on the assessments, said Chester E. Finn Jr., the board's chairman, the plan will enable NAEP to measure student performance against agreed-upon standards of what students should know and be able to do in grades 4, 8, and 12.
Under the plan, the board will report the proportion of students who performed at "basic," "proficient," and "advanced" levels at each grade.
"NAEP will report, for the first time in history, how good is good enough," said Mr. Finn, professor of education and public policy at Vanderbilt University. "What has been a descriptive process will become a normative process."
He added that the plan will also flesh out one of the national education goals set by President Bush and the nation's governors. In their joint statement, the chief executives pledged that "by the year 2000, American students will leave grades 4, 8, and 12 having demonstrated competency in challenging subject matter."
Michael Cohen, director of education programs for the National Governors' Association, said the NAEP standards will most likely be the vehicles for measuring progress toward that goal. But he added that the nga and the Administration are in the midst of discussions over the makeup and responsibilities of the panel that will monitor the goals effort.
"It's hard to say definitively at this time that the governors will say to NAGB [the NAEP governing board], 'You are the ones to do this task,"' Mr. Cohen said. "But I don't think there will be a problem with that."
Other observers cautioned, however, that the assessment board may face substantial technical obstacles in implementing its plan. And, they charged, the panel may not be the most appropriate agency to make such a fundamental decision.
"Nagb ought to be about the business of ... ensuring the quality and innovative character of NAEP," said Paul G. LeMahieu, director of the division of research, evaluation, and test development for the Pittsburgh Public Schools. "They're distracted from that in their quest to assume a strong political character."
What Does 282.5 Mean?
Known as the "nation's report card," NAEP has for 20 years measured the performance of a national sample of students in reading, writing, mathematics, and other subjects. The Congressionally mandated assessment is currently operated by the Educational Testing Service under contract to the Education Department.
The 1988 law that reauthorized the project also authorized a pilot state-level assessment in 8th-grade math in 1990. That pilot assessment, which is expected to be expanded in 1992 to include 4th-grade math and reading, will provide the first state-by-state comparisons of student achievement.
The plan to establish achievement standards was aimed, said one board member, Mark D. Musick, president of the Southern Regional Education Board, at making the test data more useful to state and national policymakers.
"You might say we already have achievement levels," said Mr. Musick, referring to the 500-point scale used to describe NAEP results. "But if you reported that Virginia scores 282.5, what does that mean?"
In contrast, he said, reporting that "28 percent are performing at the basic level, and 52 percent are performing at the proficient level gives you a lot more information that is meaningful to the public."
"I may be proved wrong down the road," he continued, "but putting compelling information before people is what it takes to mobilize them to do something" to improve education.
Three Standards, Not One
In deciding to establish achievement levels, the board cited its statutory authority to set "appropriate achievement goals" for each grade and subject tested.
In addition, it noted, the goals-setting efforts by the President and the governors also increased the demand for information about student performance.
"Defining what performance ought to be--and providing strong justification for the judgment used in making these definitions--will greatly enhance NAEP's central function as a yardstick of educational achievement," according to a paper prepared by the board.
Since the proposal was originally introduced in December, board members have agreed to drop the term "achievement goals" to avoid confusion with the other national goals-setting efforts.
In addition, the members agreed to establish three separate levels ofachievement, rather than a single standard for each grade. At a public hearing on the proposal in January, critics such as Albert Shanker, president of the American Federation of Teachers, said that setting a single standard would encourage schools to focus their efforts on students who performed just below the standard. (See Education Week, Jan. 31, 1990.)
Not Forced on States
Under the plan approved this month, the board will appoint a panel of educators, subject-area scholars, employers, civic-group representatives, and interested citizens, who will meet this fall to advise the board on setting achievement levels.
The panel, which will be divided into subgroups for different grade levels, will examine the test items to determine which questions students should be able to answer correctly to meet the standards. The standards include:
Proficient, or "solid academic performance for each grade tested." Students reaching this level, the board's paper states, "are well-prepared for the next level of schooling."
Advanced, or "superior performance beyond grade-level mastery." Students at this level have demonstrated readiness for rigorous college courses, advanced technical training, or employment requiring advanced academic achievement.
Basic, or "partial mastery of knowledge and skills that are fundamental for proficient work at each grade."
In determining the standards this fall, the advisory panel will have access to the test results, and may change its decisions after reviewing how many students answered a particular question correctly.
In the future, Mr. Musick noted, the process of setting achievement levels may be linked with efforts to develop test objectives.
"As [separate panels drawing up the objectives for each test] develop a consensus and recommend items," he said, "they should think of items that would measure basic, proficient, and advanced levels of achievement."
Mr. Musick added that, although the level-setting panel will be examining test items, most of which are to remain confidential, the meetings will be open to the public. In addition, he said, the governing board will hold an open meeting to review the recommendations, and then a public hearing on the proposals before adopting them in November.
Roy Truby, executive staff director of the governing board, added that states will have the option of using the new achievement levels or the existing scales in reporting the results from the trial state-level assessment. Thirty-seven states are participating in the project.
"N.agb will be offering the states a rich new way of understanding the data about their students' performance," Mr. Truby said. "We hope many of them request it when the 1990 math results are released, but achievement levels are not going to be reported for any state unless the state wants them. We're not going to force this on anyone."
Not a 'Done Deal'
Despite the unanimous vote by the board, some board members and testing experts warned that the nagb faces numerous obstacles in implementing its plan.
For example, said Antonia Cortese, first vice president of the New York State United Teachers and a governing-board member, the advisory panel and the board are not able to determine whether performance indicates preparation "for the next level of schooling."
"I don't think you can make that statement," she said. "I have a problem making it before studies are done."
But Mr. Finn responded that the panel's judgment would be based on members' experience and knowledge of what students ought to know and be able to do.
"On a 4th-grade panel," he said, "might be a 5th-grade teacher, who will be asked what in math a kid would have to master to be prepared for your class."
In addition, he noted, the board may in the future conduct longitudinal studies to determine whether students at the proficient level have in fact mastered higher-level work.
Mr. LeMahieu, the Pittsburgh test director, also said the board's plan may be difficult to implement. Although several districts and states have used similar procedures for setting standards for minimal competency, he said, few have successfully set standards for higher levels of performance.
"I tried it here, and we went to a different route," Mr. LeMahieu said in an interview. "We could get no consistent imagery [of the characteristics of above-average performers] with anything other than minimal performance."
Mr. Musick responded that the process that will be used this year is a pilot, and that the board may modify the procedures in the future.
"We believe the process will work," he said. "But it is not by any means a done deal."
Standards for the Test
The Pittsburgh research director also questioned whether the governing board is the appropriate body to set achievement levels.
Such standards, he noted, will represent the first national judgments about what students should know and be able to do, and may lead to substantial changes in school curricula.
"I have too darn many years of experience watching tests operate in the education arena and the public-policy arena," Mr. LeMahieu said. "If you set standards on the test, you set standards for schools."
But Mr. Musick responded that the NAEP standards do not purport to measure the breadth of schooling. He acknowledged, however, that the board's action is likely to generate controversy.
"Clearly, this test can't discern everything students should know," said the board member. "But we do say, 'Here are some of the things we believe they should know.' We haven't done that before. There's going to be a lot of debate over the strengths and weaknesses of our decisions."