The National Assessment of Educational Progress (naep), the only source of ongoing nationwide measurement of the academic achievement of elementary- and secondary-school students, has failed to make itself useful to state and local education policymakers. It must play a much more active role in setting national educational goals in order to justify its continuation, according to a major independent study of the assessment program to be published at the end of this month.
‘Underdeveloped’ and ‘Underused’
Measuring the Quality of Education, a foundation-supported study conducted over the last 18 months by former Labor Secretary Willard Wirtz and Archie E. Lapointe, concludes that although the federally funded, but independent, testing program is “a national asset of infinite potential,” it is “underdeveloped” and “underused,” and has had “apparently negligible influence” on education policy and practice around the country.
The study recommends a variety of changes in the program’s policy and procedures intended to make its data more understandable and meaningful to education officials, researchers, and the public. The study also criticizes naep’s “inadequate policy leadership.”
But it adds that budget cuts in recent years have left the Congressionally mandated organization unable to do enough data-gathering, research, and evaluation to continue to be effective. If funding is not increased, the report concludes, naep should be abolished.
In addition, the study proposes the creation of an “Educational Assessment Council” to coordinate assessment efforts by all types of organizations nationwide and an “Educational Assessment Center,” an independent agency that would house both the naep functions and the proposed Educational Assessment Council.
The study recommends an annual budget of $7.5 million, compared to a current funding level of $3.88 million.
The purpose of the “assessment of the Assessment,” said Frederic A. Mosher of the Carnegie Corporation, co-sponsor--with the Ford and Spencer Foundations--of the $285,000 study, was to look at the effectiveness of the testing organization with the idea of deciding under what conditions it could and should be
continued. “Is...the blunt instruction of 12 years and $64 million of experience that the Assessment is esoteric, too refined conceptually to get its message through to a public that reads on the run or to educational administrators caught in the toils of rough political process?” asks the study.
The study will be distributed to approximately 500 people, including state educators, contributors to the study, and members of Congress on education committees. The Congressional legislation authorizing the program expires in November 1983.
National Shifts
The testing program, developed by educators Francis Keppel and Ralph Tyler during the mid-1960’s, has spent some $64 million over the last 12 years charting national shifts in student achievement through regular testing at four- or five-year intervals of 9-, 13-, and 17-year-olds in ten different subjects.
It began its first surveys--in writing, science, and citizenship--in 1969, the same year its funding was taken over by the federal government from foundations and it was placed under the governance of the Education Commission of the States, the consortium supported by most of the states.
Language in the Education Amendments of 1978 provided a federal statutory basis for the assessment program. And the act shifted the responsibility for it from the National Center for Educational Statistics (nces) to the National Institute of Education (nie); policy control was given to an independent, 17-member policy committee.
The National Assessment, as the study acknowledges, is considered by testing experts to be one of the most dependable measures of nationwide scholastic achievement. It is currently collecting data for the third assessments of math, social studies, and citizenship, which will be released in 1983.
Its full-time staff of 44--down from 80 in 1975--conducts one study a year, half the number it completed a few years ago. Budget cuts have also forced it to focus on only three subjects--reading, writing, and mathematics--instead of its original 10.
The major, and potentially most controversial, recommendation of the Wirtz-Lapointe study urges the testing organization to make a concerted and visible attempt--through its process of setting objectives for test items and through the production of “interpretive” reports of its findings--to set national educational goals.
“In the past, the Assessment has set objectives for its test questions based on a consensus of what the schools are teaching [in a particular subject],” said Mr. Wirtz, in an interview. “They must now begin to base these objectives on a consensus as to what the schools should be teaching,” added the former Cabinet official, who was chosen by the foundations to conduct the study in part on the basis of his leadership in a 1977 College Board Advisory Panel studying the declining Scholastic Aptitude Test scores.
‘Options To Choose From’
But the study also states that the testing organization must stop short of setting national educational standards or a national curriculum. Says Mr. Lapointe: “We are unequivocal in asserting that standards have to be set by local or state agencies; but we do see the Assessment offering some options to choose from and ways of measuring them.”
And, he adds, “naep is in a unique position to attempt to provide some vocabulary for thinking about and discussing standards that are different from minimum standards.”
So-called “minimum-competency” examinations, which usually test only a student’s grasp of rudimentary literacy skills, have in recent years been introduced in 39 states.
P. Michael Timpane, dean of Co-lumbia University’s Teachers College, former director of nie, and a consultant to the Wirtz-Lapointe study, said the recommendation on setting educational goals is a “very positive development.”
“Because of the localized decision-making structure in education,” he said, “there is no really good way to look at educational issues nationwide. The Assessment...can begin to play a central role in encouraging this debate.”
The study points to the recent naep report, “Reading, Thinking and Writing,” and the just-released assessments on music and art (see story on page 8) as evidence that the program has already begun to move in the direction of national educational goal-setting. In those reports, panels of independent “experts” in each field were asked to analyze the significance of the test results.
Roy Forbes, naep’s director, said he endorses the recommendation about national educational goals but distinguishes between objectives and standards and does not favor having the organization set standards.
‘Constructive Criticism’
He described the study as “constructive criticism.” But he said he objects to its conclusion that the testing organization has failed to transmit its findings adequately to state and local educators. And he asserted that naep’s policy committee, contrary to the study’s claim, is “workable.”
Mr. Forbes said he initiated the idea of an independent review of the testing organization in 1979 in response to criticism by nie officials that the Educational Commission of the States was unwilling to change naep’s design in order to make its data easier to use and that the commission had failed to deal adequately with the larger question of the future purpose of naep
nie expressed this criticism in the course of reviewing a proposal from the inter-state agency to continue administering the National Assessment. Mr. Forbes said his suggestion for an independent review of naep gained support within nie for the Education Commission of the States’ grant proposal.
After Mr. Forbes gave nie an assurance that the review of naep (what was to become the Wirtz-Lapointe study) would take place, the federal agency “reluctantly” agreed to award the commission the naep grant, according to Gregory R. Anrig, president of the Educational Testing Service, who, as Commissioner of Education in Massachusetts, was hired in 1979 by nie to review the proposal.
Initiate the Review
Mr. Forbes said that he was never told by anyone at nie that he would have to initiate the review in order to win the grant.
Philip B. Swain, chairman of the policy committee and a member of the Washington State Board of Education, said he does not think the committee will take a defensive position on the study.
He admitted that “there is potential for greater service [by the Assessment] to the states,” but said that “making the National Assessment a household word is not an attainable goal.” He defended the “workability” of the current policy committee structure.
The Educational Assessment Council proposed by the study, would be a completely independent, six-to-eight member group that would coordinate the organization of all assessment data collected at the state, federal, and local levels, as well as ensure more useful analysis and interpretation of such information.
Made up of the nation’s leading educators, it would have no “authority” other than independent observation, would be supported by a small staff, and would be primarily concerned with ensuring that all assessment information is “translated” into a useful form for practicing educators.
Mr. Anrig says of the proposed Council, “It would ask the ‘so what’ question of assessment information: what does it really mean, what are the reasons behind the data.”
As well as being part of the 1979 panel that reviewed naep for nie, Mr. Anrig was also deeply involved in the Wirtz-Lapointe study and was one of a group of seven individuals who served as an advisory committee. Seventy other educational leaders from around the country submitted responses to a draft of the study.
Widespread Demand
All of its recommendations, the report asserts, are based on the assumption that the national program’s future is dependent upon its ability to identify itself with the current widespread demand for educational standards and accountability. The program must also, the report concludes, base its future policy decisions on an assumption that responsibility for strengthening education will be placed “squarely” on states and local communities.
Among the specific recommendations designed to make its test data more accessible to state and local educators:
That arrangements be made to facilitate the use of test information by state and local agencies for comparisons with nationwide student achievement levels.
That its tests be administered in the future on a grade-level basis at the fourth, eighth, and twelfth grades.
That the National Assessment’s procedures and capacity for developing test items be revised to meet the need for items that will measure with maximum accuracy, and in a variety of ways, students’ proficiencies as they relate to identified educational objectives.
