Before becoming president of the Educational Testing Service last September, Gregory R. Anrig was for over eight years commissioner of education in Massachusetts, a job he quit, he says, after fighting a losing battle on behalf of the state’s schools to defeat Proposition 2. (“The effect of that vote has been devastating on the school system in the state of Massachusetts.”... For the first time in eight and a half years, I was having trouble shaving in the morning and going into work.”)
ets has a $130-million budget and non-profit tax status. Last year, its tests were taken by 6 million people, who ranged from would-be podiatrists and foreign-service officers to preschoolers. In recent years, it has been attacked for being too powerful and too secretive. Mr. Anrig, however, assumes the leadership of the organization as a strong supporter of truth-in-testing laws. In his first 11 months, he has completed a major internal reorganization of ets, reducing the size of his staff by 400, to 3,100. And, for the first time, ets invited an outside panel--including a longtime ets critic, Terry Herndon, executive director of the National Education Association--to evaluate the organization’s efforts to meet its newly endorsed public-disclosure standards. Thomas Toch recently spoke with Mr. Anrig in his Princeton office about his views on testing, his reaction to the recent criticisms of ets, and his own goals for the nation’s leading testmaker.
Q:You’ve been a teacher, a principal, a superintendent. For a long time, you were an outsider looking in on ets What’s your view on the kind of testing that ets does?
A:As commissioner [of education in Massachusetts], I felt we did too much testing. Generally, I think people turn to testing too quickly and too definitively. And I haven’t changed my view on that since coming here. But I don’t want us to lose sight of the fact that tests are an important part of American education. But only constructively so to the extent that they are kept in perspective. We have an obligation at ets to keep that perspective before the public eye. And I spend a lot of time doing that.
Q:Do you think ets met this obligation in the past?
A:I do. But the times were very different. Testing was not as hot an issue in 1958 as it is now.
Q:Why not?
A:Well, education itself wasn’t as hot an issue. I think we’ve just come through a period of time where the Republic’s concern on one hand and its disenchantment on the other have just grown astronomically through the 60’s and 70’s.
Q:How does testing fit in here?
A:Some of it has to do with performance. Tests have been the report card. I think people who are angry about taxes use test results--certainly this was the case with Proposition 2. When we were out campaigning [against it], sat test scores and others were always being brought up--and they were frankly being brought up as a way of rationalizing a genuine economic strait that a lot of taxpayers in Massachusetts were in.
Q:You’re saying that tests are being used for purposes they are not designed for. As you have just suggested, the sat and other tests have been used politically, to condemn the performance of schools, when, in the case of the sat, they are only supposed to be a predictor of first-year achievement in college.
A:I think that’s true. On the other hand, the world of politics is a funny one. People are always looking for quick evidence. And because education has always been pretty poor in providing objective evidence about its performance, people grab onto anything that is there with a strength that it doesn’t warrant. And I think that is true of the sat. People, including the media, grab it so strongly, because it is the only thing out there that is across the board, nationwide, annual, and large-scale. I would like to see a stronger National Assessment [of Educational Progress]; then maybe people wouldn’t turn to the sat quite so fast. After all, over half the kids are not going on to college, and the sat doesn’t deal with them. And the states’ basic-skills tests are, by and large, at a much lower level. So you have this gap between the basic-skills test, which is basically functional literacy, and the sat, and there is nothing in between that is really giving a report card.
Q:Yet, both of those types of tests--the basic-skills tests and the sat--are being used by the media, politicians, and educators to describe the performance of students across the range of academic ability.
A:That’s right, that’s right.
Q:What do you see as the significance of the much-discussed--in the media and everywhere else--decline of sat scores over the last 14 or 15 years?
A:
[The decline] is a combination of two factors. One was the broadening of the candidate pool. More women and more minorities are taking the test every year. If that means that [average] scores are going down, that’s [a result of greater participation]. The other part of the decline was a genuine decline in performance. I can’t get exact as to the proportion of that decline.
The overall decline, by the way, is not that much; it’s about 6 or 7 percent. What’s important is that it has been a steady decline over time. It is especially important to note that the performance of the most able students has gone down. It is very important for educators not to deny this decline. Now, if we deny that, the public is going to damn us.
I don’t think that the answer is to have a prep course for the sat I think that you should go back and look at the curriculum. [If] you take a look at the high-school curriculum in English, for instance, you’ll find, I suspect, that the thickest part of the high-school catalog will be in the English department. What we’ve done over time in terms of the high-school English curriculum is chopped it up into little pieces and so we’ve got all these little courses, a quarter of a course or a semester long, and a student goes from one teacher to another teacher, and from one subject to another subject, and there’s never any continuity. The result of that, I think, shows up on the sat scores. If you look at the math section of the sat, you have to compare scores with the number of math courses students are taking in high school, and that is increasingly on the decline also. By the time that they take the sat in their senior year, it’s been two or sometimes three years since they had their last math course. Of course they’re not going to do as well.
Female students had not been taking much math. Now, they’re taking more math and they are starting to do better on the math part [of the test]. So we have to go back and look at the curriculum, I think, when we look at this decline. I think it’s very important for educators to look at that seriously, as indeed most of them have.
Q:sat results are also being used to evaluate the performance of schools.
A:That’s right. The state basic-skills tests don’t even touch the area of reasoning skills. The sat does, but not clearly. That is a whole area of learning that is very, very important that no one knows anything about in terms of performance.
Q:Are there any instances where you feel the test is being misused?
A:... I got a call from a state the other day that wanted to use the sat as the determining factor for admitting sophomores into teacher-training programs at the state university. I told them that if they did that, it would be a misuse of the test and informed them that I would say so publicly. It is my intent that, if I can identify clearly documented misuses of the test, we and our client boards together will take action against them.
Q:People often complain that ets, with its virtual monopoly on admissions testing, has become a gatekeeper, and that no organization should rightfully have that kind of power. How do you feel about this?
A:It’s a misunderstanding of the current state of college admissions to feel that anyone’s a gatekeeper. ... Today, most kids who want to go to college go to college--and to the college of their choice. The selectivity that was going on in the 50’s is just no longer a fact in American education. You still have your 100 selective institutions, but there are almost 3,000 other institutions out there.
Q:Does ets play a role as gatekeeper to the 100 most selective schools in the country?
A:Then the question is: Are the instruments that you’re using as high-quality, bias-free as any instrument of that sort can be? And in that case, I say ets can stand proudly. If you’re going to have a large organization that’s developing most of the tests in the admissions area, then you want that to be done well and fairly.
Q:Christopher Jencks and James Crouse have argued recently that the sat is in fact not an aptitude test as ets claims it is, but rather an achievement test in vocabulary, reading comprehension, and basic math. And they criticize ets for labeling the sat an aptitude test, because it does not measure, as that label implies, an applicant’s ability to learn, but rather what he has already learned, which can be determined by a whole host of factors beyond a student’s control.
A:The latter part I agree with. The label “aptitude” goes back to 1926 and I wasn’t around at that time, so I don’t know why they chose it. If you take a look at the test itself, it’s not an I.Q. test; it’s made up of vocabulary and math ... The public knows that the sat measures something that you learn in school.
Q:That would make it an achievement test.
A:Sure, so part of what Jencks and Crouse are saying is correct. We produce the achievement tests, so I have no trouble with achievement tests. The problem with their proposal is that they’re saying that they want to use testing as a way of reforming the high school. That underlies their proposal. And, one, it won’t work; two, it’s the wrong way to do it.
Q:Why?
A:It won’t work because I know of no change that can be imposed from the outside. ... What you would do [by replacing the sat with achievement tests] is determine the curriculum of the high schools in the United States. There’s no question at all that if you want to have an American History achievement test or a biology achievement test, that you then shape your curriculum so that you will do well on that achievement test.
The advantage of the sat is that it doesn’t throw off your curriculum. It doesn’t force the 25,000 or so secondary schools in the country to have the same curriculum. On the other hand, it gives you a way of measuring how those kids do from that many different places, each of which assigns its own curriculum. I don’t think there should be a national curriculum.
Q:Jencks and Crouse would argue that you could measure what a student learns in school more effectively by doing away with the sat and just offering achievement tests in a variety of subjects.
A:That has the danger of the thing that ets has been accused of--elitism. About 300,000 students take the achievement tests each year. They tend to be the most able, going on to the selective institutions. The Ivy League requires one or two of these tests to complete an application. I don’t think that that is as good as a general, nationally administered admissions test, and I think we would find that we would uncover a new set of inequities by using achievement tests in the place of the sat The new set of inequities would be this: Because they are curriculum-specific, if you didn’t happen to go to a school that had that curriculum, or where the teaching was not as good as in other schools, I think there would be some real danger of inequities.
Q:The College Board [in its Educational EQuality Project] goes a fair way in recommending something like a national curriculum, doesn’t it?
A:You have to be very careful about that. First, I’m very supportive of the project. It’s one of the most constructive efforts to reform the high schools that’s going on. What they’re doing is saying, ‘Regardless of what the courses are, the outcomes you should have, if you aspire to college, are the following'; and then they try to spell that out rather specifically. And what that’s getting at is the issue of standards and expectations-- and that, I think, is the major need of high-school reform in the 1980’s, because now it is so unclear what we expect of the high schools. Until we get that straightened out, it will be very difficult for high schools to say, ‘how can we respond to it?’
The reason I like Project EQuality, now called the Educational EQuality Project, is that it deals fundamentally with the issue of standards and expectations without saying that you have to have this course or that course. Whichever course you take, this is what you should come out of it with. That’s different from saying that you have to take the biology course that will meet the achievement test.
Q:Just what, then, is the sat designed to do?
A:The sat is designed to do one thing and one thing only, and that is to give an objective estimate of how a student will perform in his or her first year of college, no matter where that student has come from in terms of region, or state, or size of school, or kind of school. It does that. It does that quite well. That’s all it does.
It doesn’t tell you whether you’d be a good student, how well motivated you are, or whether you are going to forget about your work when you get in a dormitory. It simply makes an estimate. It’s not the best estimate available to admissions officers. The best estimate is the high-school transcript. The next best estimate is the sat The best estimate of all is the two of those together, and that means that you have less of a chance of making a mistake about a youngster.
The sat plays a very important role. If you use it as a sole deciding factor for any decision, then it’s a misuse of the test and I’d be the first to say that.
Q:In a report that came out about 18 months ago, Ralph Nader and Allan Nairn, besides pointing out a direct link between family income and higher sat scores, concluded that the test’s validity as a forecaster of academic success is scarcely more trustworthy than “random prediction with a pair of dice.”
A:The best thing I could refer you to is Lloyd Bond’s article for the National Academy of Education. As you know, I am not a researcher or a tester. Lloyd Bond is, and he’s very reputable. He happens to be black. He has written a critique of their conclusion, explaining that they were absolutely incorrect. Their computation was wrong. So what they’ve done is flawed on its face. But I cannot explain it. I read his article and I still cannot understand it. However, it is important to keep the test in perspective. By itself, the sat is not the best predictor. But combined with the high-school transcript, it is the best predictor of all. It still is not foolproof, but it is better than rolling the dice, by a long shot.
Q:The study two of your people here just released on the admissions policies of nine selective colleges concluded that the admissions people at those schools still base their decisions largely on a combination of grades and sat scores. They lamented this, arguing that there should be more emphasis on non-academic things. How do you feel about that?
A:I agree with the recommendations of the researchers. Ironically, in those colleges, they thought they were giving greater weight to other qualities. It is much to their credit that they were willing to do the research and, secondly, that they were willing to release their names with it. Because, in effect, it says that what we thought isn’t true. I find, ironically, in the time that I have been here, that I spend most of my time--and I don’t intend for it to change-- talking with public groups, trying to keep tests in perspective, encouraging people not to exaggerate their importance.
Q:How do you hope to make your mark at ets?
A:I think it will be on two levels. One is applying some leadership beyond Educational Testing Service, in terms of the issues of standards and expectations of schools and colleges, and how those can be reported to the public and the public can feel that they’re getting their money’s worth out of education. That’s something that’s going to be a prevailing theme throughout education through the years, as it is already.
The second is that I would like to see testing turned more in the direction of instruction. I can get very excited about that. Most testing is a one-time experience with one-time feedback and it’s about a kid. The youngster doesn’t use it and frequently, unfortunately, the teacher or professor doesn’t use it. What I’d like to do is find a way to provide feedback that a student, and a student’s parent, and a student’s teacher can use for instruction.
Q:If the College Board’s Educational EQuality Project gets to the point where schools start making changes in their curriculum to respond to it, do you see yourselves at ets being in a position to help these schools through this new use of diagnostic tests in two or three years from now? Is that something that you are thinking about?
A:Yes, I would hope so. I would like to see testing and instruction come closer together. And the kind of thing that Educational EQuality is now doing lends itself to that, because it clears up the issue of standards and expectations. If you have standards and expectations that are clear, then the question is always, ‘How am I doing, how close am I, how far do I have to go?’ That is what diagnostic testing should help both the student and teacher know.
Whether [our diagnostic-testing program] is tied to the Education EQuality project or separate, I think they will be very compatible and supportive of each other. And we want them to be.
Q:But the bread and butter of ets are the sat, the lsat, and the gre Are you suggesting that you are going to somehow change the nature of those tests to allow them to be used diagnostically?
A:No, probably not. Those tests serve a good purpose and they’re designed to do one thing. The kind of thing that I’m calling diagnostic testing is a totally different kind of approach to testing measurement and teaching.
What I plan to do, and I hope to do, is to be earning enough funds in the whole range of things that ets does; then we can afford to really concentrate money on bringing testing and instruction closer together. A profit-making organization could not make that decision, but we can, and indeed I think we have an obligation to.
