Richard Phelps was the project manager and author of the G.A.O. report. He now works as an analyst at Pelavin Associates in Washington. The views expressed here are his own and not those of Pelavin Associates or the General Accounting Office.
A new study by the General Accounting Office goes a long way toward clarifying the debate (Student Testing: Current Extent and Expenditures, With Cost Estimates for a National Examination). The bulk of the data for the research came from a survey of all state testing officials and a nationwide sample of local school district officials. In analyzing the responses, I was able to evaluate some frequently cited arguments against national testing:
- Weak Argument #1: There’s Too Much Testing Now. Some strong language has been employed to convince us that our students are overtested. One report a few years ago asserted that U.S. students “are subjected to too much standardized testing,’' and that testing “devours’’ teaching time and “looms ominously’’ in students’ lives. For example, it claimed that “mandatory testing consumes some 20 million school days annually.’' Twenty million is a big number. But there’s also a big number of U.S. students--about 40 million. The average is just one-half day of testing per student per year.
I found an average close to that, about 3.4 hours per student per year. The time devoted to standardized testing doubles if one counts all test-related activity. But, even one day per year would not seem burdensome to most citizens; or even to most students, since three-quarters of all standardized tests are for low stakes.
- Weak Argument #2: A National Test Would Cost Too Much. Estimates for the potential cost of a national exam have varied widely. At the low end, some test proponents have argued for a national test modeled on the Armed Services Vocational Assessment Battery and costing as little as $3 million a year. At the high end, test opponents have extrapolated from the cost of Advanced Placement exams an estimate over $3 billion a year--one thousand times higher!
I estimate that the most likely type of national exam--a multi-subject performance exam--could cost around $330 million. But, only two-thirds of this would be new costs; the rest would be compensated for by dropping some current testing. If a national exam were exclusively multiple-choice, the cost would be much less. Only under extremely unlikely conditions would a national exam’s cost exceed $1 billion.
Putting these numbers in a larger context, a national test costing about $330 million a year would amount to less than one-half of 1 percent of all spending on local public schools.
- Weak Argument #3: Clash With Local Curricula. The number of local U.S. school districts with their own official curricula can be counted on one’s fingers. Seventeen states have no set curriculum and, in another 10, local districts are not required to follow their state’s curriculum. Only 14 states both require that local districts follow a state curriculum and administer a statewide test. Most state testing officials assert that the more limiting terms “guidelines,’' “standards,’' or “objectives’’ more accurately describe the criteria on which their tests are based than does “curriculum.’'
Given that most teachers rely on textbook content for course content, and given that the large majority of textbooks are nationally distributed, how local could curricula be in practice? Sure, some states require unique course content. Requirements for social-studies courses on a state’s history or constitution provide an example. But, that’s a pittance.
- Weak Argument #4: A Serial Development Process. Some national test opponents assert that a valid national test will require a serial development process that starts with the development of a new curriculum, then with new textbooks, then with pilot tests, and only then with test production and administration. They would have us believe that one step in the process cannot start until a previous step is finished. For a national test, such a process could take over 10 years.
Of seven U.S. states that have developed statewide multi-subject performance tests, none used a serial process. All developed their standards and their tests simultaneously; the two are likely to fit together better by doing it that way. On average, statewide performance tests took 14 months to develop to the pilot stage, and 27 months to reach final form.
Phone calls to math-text editors at the five largest textbook publishers revealed that they, too, do not wait for others to finish before they start their work. Publication of the National Council of Teachers of Mathematics’s national standards, for example, did not catch them by surprise. They were ready with changes for their textbooks by the time the standards were completed.
- Strong Argument #1: Teacher and Administrator Antipathy. Any feasible national test would certainly be administered in the same schools our students attend--and by their teachers and administrators. The survey revealed considerable opposition from some of these administrators to a national test in the abstract. That is, without knowing its characteristics in advance, many feared the worst. This opposition should give pause to national-exam advocates who assume local cooperation.
All but one of six states (with 11 tests among them) and several Canadian provinces with multi-subject performance tests employed teachers and local administrators in standard-setting and test development, administration, and scoring. Doing the same for a national exam could help by giving these professionals some say over the quality of the test and the use of the results. Teacher involvement in test development, moreover, seems to strengthen adherence to standards.
- Strong Argument #2: Low Test Quality and Misuse of Test Results. The survey results show clearly that above all else respondents were worried that a national exam might be of low quality and the test results might be misused. Indeed, tests aren’t inevitably of high quality; measures must be taken and resources committed to assure that they are.
Likewise, some states and Canadian provinces have learned to handle the seemingly inevitable problem of misuse of test results. Journalistic ignorance about testing can be managed by judicious timing of the release of the results and by interpreting results for the press and the public upon release. An adequate and accurate explanation of test results should be provided to students and parents, too, and not be left to chance.
- Strong Argument #3: Interference With State Testing Programs. Many state testing programs are now well-established or soon will be. Several others are being planned. Some of these programs are large and sophisticated, and a national system will, inevitably, affect them. It should be remembered, however, that few people still propose a mandatory national exam. In a voluntary system, states would be able to decide themselves whether or not to adopt a national exam.
In part as a response to this concern, some have advocated a national system of “clusters’’ of exams that would accommodate and incorporate some of the current state tests. There’s some concern, however, that test scores across the clusters may not be made comparable, and national test-score comparability is one of the primary arguments for national testing. But, people are working on this and the jury’s still out.
- Strong Argument #4: Crowding Out Other Testing. All systemwide testing in the United States now costs about $516 million a year. A single national performance test might cost about $330 million. Of that, $209 million would be new costs, while $121 million of current testing would be dropped. Thus, a single national test could account for $330 million, or 46 percent, of a new national budget for testing of $725 million. If any single firm were to control 46 percent of all systemwide testing with a single test, it would surely assume a dominant position in the market and, perhaps, eventually a monopoly position. There may be some disadvantages to the free-market American system of standardized testing, but there are also many advantages and these could be lost.
Some testing experts would argue, further, that no matter how fair a test is, it is simply unfair to make high-stakes judgments of students based on any single test taken at just one point in time. With this concern in mind, some states and Canadian provinces use only “blended scores,’' or multiple indicators, to make high-stakes decisions. Multiple indicators can include more than one test score, grade averages, and other data. Though a national performance test may not cost as much as some had feared, it will still cost a lot in relative terms. And it may have the effect of crowding out other tests--other valid indicators of student performance.
- Strong Argument #5: Test Security May Not Be Adequate.
