The Reagan Administration has been much criticized for its selection of new names and faces to review funding proposals submitted to the National Institute of Education (NIE), other units of the Education Department, and such agencies as the National Endowment for the Humanities (NEH). It is said that the Administration is “politicizing” the peer-review process, that it is choosing evaluators who are not “expert” in the fields covered by the proposals, and that in some cases it is recruiting reviewers who are activists rather than scholars.
There are not many facets of this Administration’s handling of education that I would defend. But a word must be said in support of the principle of enlisting new reviewers, if not necessarily on behalf of every individual who has been thereby recruited.
This is no small matter, for the peer-review process is fundamentally important. The “social” agencies borrowed the process from the “science” agencies such as the National Science Foundation and the National Institutes of Health, where over the decades enormous sums of money have been apportioned among tens of thousands of applicants with the advice and assistance of outside experts.
When it works well, the peer-review process serves four purposes:
The nature of quality control changes dramatically, however, in an “unsolicited grants” competition such as NIE has recently conducted, or when schools and school systems are seeking funds for highly variegated activities within the broad rubric of “the humanities,” as was the case with a recent round of NEH proposals. Here the agency is faced with a classic “apples and oranges” (and bananas and kumquats) situation. Rather than picking the best among essentially similar projects, someone must decide both whether a given applicant is competent to do what he proposes and whether what he proposes is more important, worthwhile, or valuable than the very different projects that others want to conduct with the same, inevitably limited, funds. The agency has a Congressional mandate to buy fruit, so to speak. But who is to say whether the nation would be better off if it invested scarce resources in a plum, three grapes, or a slice of watermelon?
In such circumstances, the outside-review process evolves, at least in part, into an extension of Administration policy making and priority setting. And the reviewers become, at least in part, extensions of the Presidential appointee who heads the agency. It is altogether reasonable for him to select reviewers who share his values, his instincts, his convictions, and his hunches. In this sense, the peer-review process is necessarily and properly “political.”
Particularly in the early months of a new Administration, and especially when that Administration believes it was elected to make major shifts in federal priorities, the agency head is more apt to find kindred views and parallel values among carefully selected outside reviewers than within a career civil service imbued with the policy emphases of the previous Administration. He is certainly more likely to find them among reviewers he selects than among the panels named by his predecessor.
There are attendant risks, to be sure. The gravest is that the agency head will select ideological soul mates who do not know enough about the agency’s overall mandate to recognize what is and is not within bounds; who have too little experience with the subjects at hand to be able to distinguish fruits from vegetables; or whose awareness of the ''state of the art” in a particular field is so scanty as not to be able to judge whether a given project is competently designed.
Based on my own experience with a recent NEH panel--yes, I was one of the new reviewers--those risks are easily contained. My dozen or so colleagues were, without exception, intelligent, savvy, and discerning. They managed both to assess the quality of the proposals (which was distressingly poor in most cases) and to make reasoned judgments about which projects were worthwhile and important. Few university departments are as rigorous, as quality-minded, and as objective in judging the work of their own faculty--or of their students!
I also evaluated a batch of “unsolicited” proposals that were received by NIE and parceled out among reviewers, many of them new to the agency rolls. Since NIE “field readers” do not come together in “panels,” I have no idea who else read these proposals or what they thought of them. (Most of those I read did not deserve the taxpayer’s support.)
But it is not reasonable to fault the agency for picking new reviewers when it expects them to base more than a third of their judgment (a maximum of 35 out of 100 “points” awarded per proposal) on the “significance of the proposed research for American education.” For who is to say what is significant for American education? Opinions differ, and they should. When the demand for funds exceeds the supply by more than 10 to one, the NIE director--or any other agency head--would be violating this trust if he did not seek the advice of people whose opinions about educational “significance” he deemed congruent with his own.
But opinions alone are not enough for a credible peer-review process. Another 35 points were to be meted out according to the “quality of the proposed research project” (that is, the soundness of the methodology, the likelihood of successful completion, the applicant’s familiarity with “pertinent previous work”), 20 points were to be awarded according to the professional qualifications of those who would be carrying out the project, and so forth. Qualitative judgments must be made, as well as value judgments, and it would be wrong for an agency head to rely on reviewers who cannot make sound ones.
That does not mean, however, that he or she must look only to persons with a long history of direct involvement with a particular line of inquiry or set of activities. The classic criticism of the whole peer-review system, voiced long before Ronald Reagan and his team arrived in Washington, is that it encourages logrolling by asking members of a particular fraternity to judge proposals from other members, knowing full well that a year or two later they will switch places and be judged (and rewarded and punished) by those whom they had previously been judging.
Those who belong to such a fraternity, and who benefit from its mutual beneficence, are naturally disposed to resist the intrusion of outsiders into the review process. They are likely to fight with the most effective tool at their disposal: the charge that the newcomers are not “qualified” to make “expert” judgments about the work of long-time insiders. The implication, of course, is that the reviewers lack the specialized knowledge to make objective, professional evaluations. But what may in fact be meant is simply that the new people have other priorities and do not take for granted that longtime fraternity members ought to continue to receive support for their accustomed activities.
Of course an agency head could set out to choose reviewers who would help him carry out vendettas against particular applicants or types of activities, and he may flaunt their “expertise” to legitimize his own quixotic preferences. But a great many funding decisions are, in the final analysis, at least partly subjective. They are partly political. They hinge on questions of what is worthwhile and important; questions of whether limited funds should be used for elementary-school philosophy or high-school literature, for women’s studies or school-effectiveness studies, for profound scholarly inquiries into the nature of learning or for manuals that teachers can use in preparing multiplication lessons; questions about the relative importance--this year, for this Administration, in this country--of one issue or another, of pears or grapefruits or blueberries.
The Reagan Administration is well within its rights to pick its own reviewers, as would be any other Administration. That this Administration has distinctive, strong, and sometimes eccentric ideas about education does not make a particle of difference. But the education and research communities are also within their rights to scrutinize the process. If authentic incompetents are chosen, the cry should go up. That is part of the genius of the peer-review system. But disagreements about basic social values and educational priorities cannot be resolved through peer review, powerful though it is. They must be handled in the voting booth.
