As the House and Senate conferees in Congress finally meet to hash out seemingly small but devilish details in their respective responses to President Bush’s education plan, it seems likely that the most far-reaching provision—the mandatory testing of every 3rd through 8th grader every year—will survive intact. While the states’ experience with high-stakes tests has so far been varied, complex, and often stressful, we believe that, done properly, there is an important role for these programs in strengthening public confidence in our schools.
The testing industry has so far had few incentives to ensure that its products and procedures can withstand reasonable stress or scrutiny.
For this role to be fulfilled, however, Congress must also ensure that the testing industry is able to live up to its new importance in the lives of students and schools. A recent series of critical lapses in the construction, administration, and scoring of high-stakes tests affecting students in more than 20 states underscores issues that have troubled knowledgeable observers for years. The testing industry, dominated by a handful of unregulated companies, and on whose performance the educational careers of millions of American students increasingly depend, has so far had few incentives to ensure that its products and procedures can withstand reasonable stress or scrutiny.
In fact, it has a history of resisting just this sort of accountability. In 1981, New York state Sen. Ken LaValle proposed the nation’s first “truth in testing” legislation, a bill that proposed regular disclosure of the contents and scoring procedures of each edition of the SAT administered in New York. The College Board and the Educational Testing Service (owners of the SAT) lobbied fiercely against it, spending millions of dollars to argue that students had no right to determine whether or not their answers were scored correctly and predicting the crippling of the college-admissions process.
Congress must ensure that the testing industry is able to live up to its new importance in the lives of students and schools.
A few years later, the admissions process had not collapsed, the tests had fewer overtly discriminatory or otherwise flawed questions, and even the ETS had agreed that the law had been a great success.
Over the past six years, almost every state has introduced mandatory elementary and high school tests; President Bush’s plan would make that mandate a national one. Since poor performance on these tests has serious consequences for students and educators, they need the same sort of openness that truth-in-testing brought to the college and graduate school admissions process.
As superintendents and state departments of education across the country have discovered, market forces alone cannot protect the integrity of this crucial process when most of the information required by the states and districts to make informed decisions is held by the testing companies and disclosed only at their pleasure. Furthermore, an entire class of “customers"—students, parents, and citizens—has effectively no access at all to the information, and hence no way to participate in some of the most important decisions with which schools are faced. Clearly, this situation will only get worse as the number of tests and test-takers expands drastically over the next few years.
We believe that a national truth-in-testing provision is called for to prevent calamities on a much larger scale, and have proposed that such a measure be added to the president’s education legislation now being considered by Congress. We suggest that each time a test is given pursuant to federal requirements, the testing agency be required to disclose for timely public inspection relevant information regarding the construction of the test and its items, as well as the actual questions and answers used, a description of the formulas used to convert raw responses to scaled scores, and a disclosure of any claims made by the testing company about the accuracy of scoring and the timeliness of reporting. In addition, the testing agency would have to give a breakdown of significant differences in response rates by different ethnic and socioeconomic groups. Testing agencies would also be required to explain in easily understood language what steps have been taken to ensure that the test is fair to test-takers of all backgrounds, and to specify what steps may be taken to challenge a score.
Over the past six years, almost every state has introduced mandatory elementary and high school tests.
These provisions essentially mirror those of Sen. LaValle’s original bill, which has worked so well for the people of New York.
We need trustworthy accountability systems to help restore faith in our public schools. Open disclosure has proven to be an effective method of ensuring fair practices without intrusive regulation. The U.S. Securities and Exchange Commission and GAAP provisions (outlining “generally accepted accounting principles”) are the bedrock on which we have built the world’s most extensive and democratic financial markets. Standardized disclosures for real estate sales and mortgage loans provide information and protection for all parties.
Why should the “market” of educational opportunity have any less protection? And why should the testing companies, on whom we rely to bring accountability to our schools, themselves be unaccountable?
John Katzman and Steven Hodas are the co-authors of Class Action (Random House, 1995) and the co-founders of Homeroom.com. Mr. Katzman is the founder and chief executive officer of The Princeton Review.
