The surprise release last week of a study questioning the validity of Texas students’ rising scores on state exams has stirred heated accusations and countercharges in the waning days of the presidential race.
Vice President Al Gore said the report proved that the “Texas education miracle” touted by his Republican rival was nothing more than a myth. Gov. George W. Bush of Texas disputed the findings and questioned why they had been released so close to the election.
Some prominent researchers who have studied the Texas achievement record also questioned the findings.
At issue are the test-score gains made in the 1990s by Texas students—blacks and Hispanics as well as whites—on the Texas Assessment of Academic Skills, the state’s high-stakes testing system.
For More Information |
| Read the new RAND report, “What Do Test Scores in Texas Tell Us?,” as well as the think tank’s earlier study, “Improving State Achievement: What NAEP State Test Scores Tell Us.” |
The new study, produced by researchers at the RAND Corp., found that the students’ dramatic improvements on the TAAS were not mirrored by similar gains on the National Assessment of Educational Progress, the nation’s yardstick for measuring student achievement.
“One plausible explanation ... is that many schools are devoting a great deal of class time to highly specific TAAS preparation,” the RAND research team speculates in its paper, which was released Oct. 24.
The researchers also cast doubt on the idea that Texas had succeeded in narrowing the achievement gap between white and minority students. Between 1994 and 1998, the study found, those gaps either grew slightly or held steady for Texas students on the national tests even as they appeared to be diminishing on the state tests.
Mr. Gore, the Democratic nominee, lost no time last week in using the report as ammunition against the Republican candidate. On the stump in Nashville, Tenn., on Oct. 25, he said the researchers’ conclusions showed that Texas’ test-score gains were “illusory” and had done nothing to “really raise standards or to lift up our children.”
Meanwhile, the Democratic National Committee began airing a television advertisement featuring the study’s results in key battleground states.
Bush campaign officials and some researchers were just as quick to dismiss the study.
“The progress in Texas is indisputable: Independent data shows that Texas is a national leader in student achievement,” Dan Bartlett, a campaign spokesman for Mr. Bush, said in an Oct. 24 press release. “The 14-page [RAND] opinion paper, issued by four researchers two weeks before the election, directly contradicts every credible, nonpartisan, scientific study that found Texas and North Carolina outperformed their peers in student achievement.”
The morning after the RAND release, Jay P. Greene, a senior fellow at the Manhattan Institute for Policy Research, a think tank in New York City, released an analysis of his own that was more supportive of the Texas education record.
“It’s misleading,” he said of the RAND study, “and it’s surprising that it’s come out so late in the campaign when people don’t have time to sort through the facts.”
The rapid responses by both campaigns signify the importance of education in the presidential race, which was still neck-and-neck as of late last week.
“People are paying attention, and the race is so close,” said Mark S. Mellman, the president of the Mellman Group, a Washington-based polling firm with Democratic clients. “The significance of any issue that breaks through the press is magnified.”
TAAS vs. NAEP
Ironically, the independent study to which the Bush campaign referred in its Oct. 24 press release was produced in July by another RAND researcher, David W. Grissmer. Looking at students from families with similar demographic and socioeconomic factors, Mr. Grissmer’s 300-page study concluded that Texas and North Carolina had made greater academic gains than any other states on NAEP tests given between 1990 and 1996. (“RAND Report Tracks State NAEP Gains,” Aug. 2, 2000.)
The seemingly conflicting findings from different RAND scholars, along with the timing of the new study’s release, put the respected, nonpartisan 52-year-old think tank in an uncomfortable position. In the hope of allaying criticism, James A. Thomson, the president and chief executive officer of the Santa Monica, Calif.-based organization, issued a public statement to accompany the new publication.
“The timing of the release of both reports was based on the same, constant RAND standard; we release our work as soon as the research, review, and revision processes are complete,” Mr. Thomson said in his statement. “We don’t distribute them for political reasons, and we don’t sit on them for political reasons.”
A RAND spokesman said the Bush and Gore campaigns were both notified about the report on Oct. 23, the day before its release.
“We stand behind the quality of both this paper and of our July report on the meaning of national test scores,” Mr. Thomson said.
He and the authors of both RAND reports said the findings differ because the researchers were looking at different issues and employing different research methodologies.
Mr. Grissmer was trying to explain the variation among all the states in scores over time on the national assessment. To level the playing field, his numbers were adjusted to account for socioeconomic differences in states’ student populations. He also studied a wider span of test scores, focusing on seven different NAEP tests given between 1990 and 1996.
Still, Mr. Grissmer said the new study was “premature” in suggesting that the TAAS results were invalid.
“TAAS and other state tests are narrowly designed tests that try to measure what is taught in particular grades,” he said in a written statement last week. “They are not designed to measure wider knowledge not taught in the curriculum.”
In comparison, the study released last week concentrates on Texas’ results from national assessments given between 1994 and 1998 in reading and mathematics. The researchers for the new report said they chose that period because it marked the start of the state’s new, tougher testing program, which began before Gov. Bush took office in 1995.
Because the scores are used to determine which schools fail and which students can graduate from high school, the TAAS has been a focus of educators and researchers for years. But Mr. Bush’s presidential campaign, in which he has emphasized education, has heightened the scrutiny of the state’s student- achievement record.
“So far as I know, no one had any [political] ax to grind,” said Brian M. Stecher, an author of the new RAND study. His co-authors are Stephen P. Klein, Laura S. Hamilton, and Daniel F. McCaffrey.
Mr. Stecher, a senior social scientist at RAND, said the researchers’ interest in the Texas tests was piqued in the spring when they were looking at the results from several tests, including the TAAS, from a larger experiment on teaching practices and student achievement.
The researchers speculate that the reason for differing test- score patterns between the national and state tests may be that teachers are spending large amounts of time “teaching to the test” for the state assessment.
“If you look beyond Texas to the research on what happens when there are high-stakes tests, you find that there is evidence of a narrowing of the curriculum, a decreasing emphasis on things that are not tested in favor of what’s tested, and extensive test preparation,” Mr. Stecher said. “What we know about Texas suggests these things are happening there, too.”
That message, he added, has gotten lost in the heated political debate over whether “Texas is good or Texas is bad.”
One political analyst said Democrats hoped to use the new RAND study to hurt Mr. Bush on education, one of his strongest issues.
“If you can undermine the source of his greatest strength, you can wound him seriously,” said Thomas E. Mann, a senior fellow at the Brookings Institution, a Washington think tank.
Political Impact
Not surprisingly, Democrats interviewed last week said the new RAND report would help Mr. Gore’s chances.
Mr. Mellman, the Democratic pollster, said Gov. Bush had been “somewhat successful” in winning over voters on an issue that is more commonly associated with Democrats. “From the get-go, in different ways and under different guises, he has tried to narrow the gap in education [between the two parties],” Mr. Mellman said.
On the basis of the new study, the pollster asserted, “it looks like he’s narrowed the gap on a fraudulent basis.”
A Republican pollster, meanwhile, said the report would have “little or no impact” on voters. She maintained that Mr. Bush has effectively communicated his vision on education since the beginning of the 2000 campaign.
“It’s very difficult for Gore to make significant inroads on this particular issue,” contended Linda A. DiVall, the president of American Viewpoint, a Republican polling firm in Alexandria, Va. Voters see Bush “as someone who has core beliefs in this issue,” she said, “and they are not going to be swayed.”
—Staff Writer David J. Hoff contributed to this story.
