Opinion
Assessment Opinion

What do PISA and TIMSS Tell Us?

By William Schmidt — April 24, 2014 3 min read
  • Save to favorites
  • Print

Today’s guest contributor is William Schmidt, University Distinguished Professor, Michigan State University; Director, Center for the Study of Curriculum.


Growing awareness of the crucial role of education in a country’s economic competitiveness has made the results of international assessments a major public event. The Trends in International Math and Science Study (TIMSS) and the Program for International Assessments (PISA) have been used as fodder for political and policy debates, in particular that the mediocre performance of U.S. students in mathematics on these tests is evidence that significant changes are required in the American educational system.

Unfortunately a lot of discussion of the TIMSS and PISA results involves hasty generalizations based on country-level averages and an obsessive focus on international rankings. Far too often we find simplistic imitation of a high-ranking country’s educational policies, or sweeping assertions that U.S. performance is determined by student poverty or the false belief that higher-performing countries only test their brightest students. The tests are also often confused with one another, partly because they happen to have similar scales with averages around 500. Yet there are important differences between the two. The different rankings of the U.S. on the TIMSS and PISA are in part due to the different countries that take each test - there are fewer rich countries that take the TIMSS, which makes the U.S. look a bit better - but the tests themselves have different content. TIMSS assesses mathematical knowledge, but PISA assesses mathematical literacy (how math is applied). Although most countries’ performance is similar on both tests, there are important exceptions where a country does quite well on the TIMSS but poorly on the PISA, or vice versa. Finally, it is important to remember that the international average is the mean across countries, not the mean of all students in every country. The latter is a bit lower because of the below-average performance of some larger countries like the U.S.

However, we should be paying a lot less attention to country averages and rankings anyway. It’s a mistake to define a country’s students based on a single number. The reality is much more complicated. In fact most of the variation in student performance on both the TIMSS and PISA is within countries, not across them. Simply because Japan’s students have a higher average test score doesn’t mean that every Japanese student does better than every U.S. student - in both there is a wide variation in student outcomes. These variations exist between schools but also within schools. We are used to talking about “good schools” and “bad schools” as if every student attending them is performing at the same level. Yet the TIMSS and PISA demonstrate that this just isn’t true. Research based on the TIMSS suggests there is also a wide variation in performance across classrooms. Unfortunately PISA doesn’t sample by classroom, concealing this important source of inequality.

The key lesson to be drawn from international assessments is that the system of education - the package of educational policies - has a major impact both on the average performance of students and the inequality among students. Student poverty is an important contributor to both of these, but U.S. performance cannot be attributed solely to the number or distribution of poor and disadvantaged students. Some countries do a much better job at mitigating the effects of student poverty, and poverty cannot explain why even affluent students trail their peers in other countries. Other nations have much greater equality in educational outcomes, and we should study their approaches and adapt them to our own circumstances.

There is a great deal to learn from international assessments like the PISA and TIMSS, and the overall performance of a country can give us a clue as to which nations to focus our attention on. However, we must take care in how we interpret the results of these tests if we are to avoid drawing misleading conclusions.

William Schmidt
Center for the Study of Curriculum
Michigan State University

The opinions expressed in Assessing the Assessments are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
iStock/Getty
Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Image of a test sheet.
sengchoy/iStock/Getty