Mr. Peiffer said Maryland would be reluctant to give up the NAEP trend data. “It would still be of use to us,” he said, “for making improvements, marking our progress.”
Beyond the political turmoil buffeting President Clinton’s plan for new voluntary national tests for students, experts are raising concerns about the proposed exams’ value as assessments.
Will the 4th grade reading and 8th grade math tests, as the administration argues, help improve classroom practices and student learning? Or will they, as some experts fear, prove useless or have unintended negative consequences?
“There is a whole history of trying to use tests to change curricula, and the record there is not particularly sterling,” said H.D. Hoover, the director of the Iowa Basic Skills Testing Program at the University of Iowa.
Much about what the annual tests, scheduled to be given for the first time in 1999, will look like is still uncertain. A national panel this month is supposed to approve recommendations for test items.
The recommendations will then be forwarded to Riverside Publishing in Itasca, Ill., which will create the reading test, and Harcourt Brace Educational Measurement in San Antonio, which will write the math test.
The companies were named by the Department of Education last month as two in a multifirm alliance to receive a $13 million contract for the first year of the planned five-year testing project. American Institutes for Research, a nonprofit corporation based in Washington, will oversee the contract.
Meanwhile, Republican members of Congress have vowed to pick up where they left off before the summer recess to try to cut off funding for the creation of the tests, which have come under fire from both the political right and left since President Clinton announced the plan in his State of the Union Address in February.
Now, some test experts are raising another set of issues, such as whether the new national tests will offer any advantage over existing commercially available basic-skills tests and whether they might inadvertently wreck the longstanding National Assessment of Educational Progress.
Some educators also worry about how the test results might be used for punishing students or rewarding teachers.
In short, how much can be gained through the tests and at what price?
One Among Many
One of the most basic questions being weighed by assessment experts is whether the new tests will provide more or better information than that gleaned from other tests designed to evaluate individual students.
“I don’t know myself what difference there is going to be between it and a good standardized off-the-shelf test,” said George F. Madaus, a professor of education and public policy at Boston College’s school of education.
Several such tests are available nationally that yield individual results, including the Iowa Test of Basic Skills, the Stanford Achievement Test, and the California Achievement Test, Mr. Hoover of the Iowa test pointed out.
“If students currently take something like ITBS or Stanford or California,” he said, “there is no new information that people are going to get [from the national tests] that they don’t already get.”
Others argue, however, that the new tests will be an improvement over commercial multiple-choice tests because half the time students spend on the national tests is to be on items that require them to write out answers. That activity, they say, is a more meaningful method of assessment.
Other experts, meanwhile, are unimpressed by the Education Department’s argument that the tests will report student scores in terms of NAEP achievement levels: basic, proficient, and advanced. More than one labeled those levels “arbitrary.”
Appealing to Some
But the national tests appeal to officials in Kentucky, one of just six states to sign on to the initiative so far and a state considered to be a leader--at least until recently--in implementing new assessment systems. (“Kentucky’s Student-Assessment Director To Leave Post,” in This Week’s News.)
In part, the tests answer the call of parents and others who want to see how individual students stack up against national benchmarks, said state schools Superintendent Wilmer S. Cody. They also fit the state legislation that calls for state tests that are like NAEP, he said. “It’s a better mousetrap.”
Eyeing the Stakes
Supporters of the new tests hope they will help hold states and schools to higher academic standards than they may set for themselves now. Boosters are counting on parallel changes in the kinds of lessons students get in class.
Some educators and assessment researchers are skeptical, though, about whether the tests will improve classroom practice and student learning in a substantial and meaningful way that helps students think better in the long term, once the tests are behind them.
Lyle V. Jones, a research professor in the psychology department at the University of North Carolina at Chapel Hill, said he fears efforts to recast classroom curricula will focus simply on teaching what will likely produce higher scores on the national tests.
“The pressures to teach what is being tested are bound to be very large and hard to resist,” he said.
“Particularly in schools where the teachers and principals know the results will be published, the focus will be on getting kids to perform well on the test rather than meeting a richer set of standards in mathematics learning,” Mr. Jones said.
Such an outcome may be especially likely, experts say, if there are significant consequences--what educators call “high stakes"--associated with test performance, such as promotion to the next grade or bonus pay for teachers.
“My real concerns,” Mr. Madaus of Boston College said, “are around the uses to which the results will be put.
“Clinton made reference to ending social promotion in the same breath as talking about this test,” he said.
“You could infer this was to be used as a [gatekeeping] kind of effort” so that 4th and 8th graders don’t advance if they don’t do well on the national tests, he said.
That’s why Lorrie A. Shepard, the interim dean of the education school at the University of Colorado at Boulder, said the Education Department ought to forbid high-stakes uses of the tests or lay out ground rules for those uses to assure that teachers have been trained to teach what is on the tests and students have had an opportunity to learn it.
In Maryland, one of the half-dozen states that have agreed to give the national tests, an education official said he would be surprised if schools used the new tests for student promotion or retention.
What’s more likely to happen in his state, said Ronald A. Peiffer, an assistant state superintendent, is that a community will hold a school accountable for the results.
Because Maryland’s innovative state assessment system was designed to transform pedagogy and curriculum, it encourages teaching with the state test in mind.
Mr. Cody also expressed doubt that the national test would be turned into a high-stakes instrument in Kentucky because the current practice there is to issue rewards and sanctions based on the performance of whole schools, not individual students.
A Threat to NAEP?
Experts also say they are apprehensive that the introduction of the new tests could harm the “gold standard” status of the National Assessment of Educational Progress.
The tests are to be based on NAEP, with the new reading and math exams using the same framework as the national assessment.
The existing assessment, which is also a project of the Education Department, is mandated by Congress and has been given since 1969.
It is the only ongoing, nationally representative assessment of U.S. student performance in core academic subjects.
Participation is voluntary, but most states take part in both the testing leading to a national score and that yielding state-by-state performance rankings.
NAEP is prohibited from providing results for individual students.
The new tests will measure individual students’ achievement in comparison with that of states, the nation, and--in the case of math--other countries.
Testing experts--even those who disparage the national assessment’s achievement levels--point to NAEP’s positive features, such as its nearly 30 years of trend data and its political credibility.
NAEP’s design is also a strong point, they argue, because it covers a wide spectrum of what students should know and be able to do in a given subject.
A sampling of students takes different parts of the NAEP tests; no one student sees the whole scope of the multihour exam. Some worry that the shorter, 90-minute length and smaller scope will make the new national tests less solid.
At least in 4th grade reading and 8th grade math, Ms. Shepard said, the new tests will come into direct competition with NAEP.
Indeed, NAEP is scheduled to measure national and state-level performance in 4th grade reading in 1998 and 8th grade math in 2000.
Because of the time, effort, and expense involved, “I can’t imagine that people [in states] would want to do both,” she said.
States do not pay for the state NAEP exams themselves, but must provide employees to distribute, administer, and collect the tests.
For the new national tests, the Education Department plans to pay for states and districts to take part in the first year of testing, but it has made no commitment beyond that.
“One thing that will clearly happen,” Ms. Sheppard predicted, is that “the national assessment will stop collecting information at the state level.”
At that point, she said, “the trend lines stop, and the value of the state assessment is now up for grabs.”
Mr. Peiffer said Maryland would be reluctant to give up the NAEP trend data. “It would still be of use to us,” he said, “for making improvements, marking our progress.”
