Participants at a session of the American Association for the Advancement of Science’s annual meeting here last week got an advance view of what is expected to be the most comprehensive and accurate international comparison ever conducted of student achievement in science and mathematics.
The Third International Mathematics and Science Study, which will be administered by the International Association for the Evaluation of Educational Achievement, will measure the abilities of students in grades 4, 8, and 12 from more than 50 countries.
Researchers told attendees that the results will not only provide a benchmark of student achievement worldwide, but will also offer greater insight into the strengths and weaknesses of different educational systems.
“We do not intend to make this a simple horse race that can be printed in USA Today to the delight of editorial writers,’' said Albert E. Beaton, a researcher with the Center for the Study of Testing who is helping develop the exam.
Although it is inevitable that the public will try to rank American students against their counterparts, panelists said, it is vital that the data collected be closely scrutinized for clues about which pedagogies are most successful and why.
“It is important when the TIMSS data come out that they be used in a thoughtful way’’ and not as an “intellectual Olympiad,’' said Dorothy M. Guilford, a National Academy of Sciences official who moderated the session.
‘Equally Unfair’ Questions
Students in the targeted grades, or the equivalent levels, will take the exam next fall if they live in the Southern Hemisphere. In spring 1995, students in Northern Hemisphere nations will take the test.
Steps will be taken to insure compatibility of student age ranges. For example, the 4th-grade test will be administered to students in the two grade levels in each country that contain the most 9-year-olds. Similar cohorts will be sampled at each grade level.
The new approach to age grouping is expected to correct a weakness of other international assessments, panelists explained.
In other internationally administered tests, the researchers said, the performance of U.S. high school students, who constitute roughly 75 percent of their age cohort, has sometimes been compared with results from countries where only the top 15 percent of students attend high school.
The test developers also have made stringent efforts to insure that the test is “equally unfair’’ to all students, said William Schmidt, the TIMSS project director.
The exam will include questions that represent “a union’’ of all the topics included in the curricula of many nations, rather than simply testing only those elements common to all.
Another element that can affect the fairness of the exam is the quality of the translation of test questions, noted Ronald K. Hambleton, a professor of psychology at the University of Massachusetts.
Cultural Differences
But cultural considerations are also important in international assessments, Mr. Hambleton said.
Some nations appear to take international comparisons more seriously than others, he said.
In one country, before a previous exam, he said, “The students were marched into the auditorium, flags waving, and told they had the honor of representing their country.’'
Test questions also can reveal cultural bias. One prototype question presented students with an illustration of a book, with a bookmark inserted at a page roughly a third of the way from the start.
Students were told how long the reader had taken to reach that point, and then asked to project how long it would take to finish if the book was read at a constant rate.
But the “correct’’ answer for European and American students was different for those from cultures in which books are read, in Western eyes, from back to front, Mr. Hambleton pointed out.
Moreover, because the test includes some performance evaluations in which students conduct experiments, the materials had to be as inexpensive as possible.
To help counter the problems created by cultural and economic differences, TIMSS officials plan to have bilingual researchers live in the tested countries for eight months. The researchers will observe and videotape school classes to get a sense of how students are taught as well as how they do on the test. (See Education Week, Feb. 16, 1994.)
