Testing experts are building a case against giving teachers and principals wide discretion over what accommodations to offer test-takers with disabilities or limited proficiency in English.
Two recent studies have found that unlimited accommodations--such as reading test questions to students or removing time limits--may give skewed views of student and school performance.
“Just because it boosts performance doesn’t mean we’re getting a valid reading of what the student can do,” said Lorrie Shepard, a professor of education at the University of Colorado at Boulder.
“It could be that we’re getting an unfair reading” that overstates students’ abilities, Ms. Shepard told her colleagues here at the annual conference of the National Center for Research on Evaluation, Standards, and Student Testing, or CRESST, at the University of California, Los Angeles.
In one of the recent studies, Ms. Shepard reviewed data from Rhode Island 4th graders who took two mathematics tests: One was an off-the-shelf standardized test; the other, a performance-based assessment. Limited-English-proficient and disabled students weren’t offered accommodations for the standardized test, but they were given extra help if the teachers felt they needed it on the performance-based test.
While students with a limited grasp of English scored better on the performance assessment than on the standardized tests, Ms. Shepard said, most of the “incredible gains” happened in the four schools that liberally offered extra help to those students.
“If we were the police, we would go to those four schools and say, ‘It doesn’t look like your practices are credible,’” she said.
The findings are in line with research on accommodations for disabled students in Kentucky, according to Daniel Koretz, a senior social scientist for the RAND Corp., a Santa Monica, Calif.-based think tank.
In research presented at last year’s CRESST conference, Mr. Koretz found that some disabled students outscored their peers because of the accommodations they received on multiple-choice tests. (“Assessment Conferees Question Clinton’s Testing Proposal,” Sept. 17, 1997.)
After reviewing new data, Mr. Koretz found that the accommodations may help on multiple-choice questions, but not open-ended ones.
“When presented with open-ended questions, the big gap returns even with accommodations,” said Mr. Koretz, who is also a professor of educational research, measurement, and evaluation at Boston College. “That’s after the same group outperformed the mean on multiple-choice tests two years earlier.”
The studies suggest that test-writers must specifically outline which accommodations should be available to which students and monitor how proctors provide those accommodations, the researchers said.
“The use of accommodations in an unregulated system is at the very least problematic,” Mr. Koretz said.
Policymakers turn to tests to judge how well schools are teaching their students and to give parents an idea of how well their children are learning. Often, though, testing experts say, states select standardized tests without any ties to their curricula.
“That has a very seductively simple appeal to it,” William H. Schmidt, a professor of education at Michigan State University in East Lansing, told attendees at the Sept. 10-11 gathering. But “what if you did this with different tests, and you didn’t get the same rankings for schools?”
Different kinds of tests emphasize different skills, Mr. Schmidt said, and their reports of scores don’t explain whether a student has mastered a specific skill or concept, such as rounding numbers in math or understanding the principles of matter in physics.
Mr. Schmidt, who was the national research coordinator for the Third International Mathematics and Science Study, found that some students from some countries excelled in certain areas of a subject while showing poorly in a separate section of the same field. In the United States, test scores revealed “huge variations across schools,” he said.
Like the TIMSS test, a standardized-test score may give a general indication of how a school or student is performing. But there’s no guarantee that an assessment that emphasizes a distinct set of skills wouldn’t show different results, Mr. Schmidt said.
He suggests that policymakers rely only on test data that show how well students perform on specific areas of a subject.
That approach, however, is politically unrealistic and will be unnecessary once states have built assessment systems around the academic-content and student-performance standards they’ve been creating, according to another researcher.
“Parents don’t care how kids are doing on a specific section of the curriculum,” said Joan L. Herman, the associate director of the federally funded CRESST, which is based at UCLA. “They want the whole answer.”
--DAVID J. HOFF
