Assessment Opinion

OECD: Poverty Explains 46% of PISA Scores

By Paul B. Ash — March 28, 2014 3 min read
  • Save to favorites
  • Print

Today’s guest contributor is Paul B. Ash, Superintendent, Lexington Public Schools, Massachusetts.

There has been much discussion lately around international exams such as the PISA, and the significance of student test results for American educational policy. This important conversation, however, is distorted when educators and politicians try to draw conclusions about the significance of test scores beyond what the test itself was actually designed to measure.

Iris Rotberg, in her adjacent article, argues that international tests of science and mathematics, such as the PISA, are not related to international economic competitiveness. She argues,

International test-score rank was only one of the 113 criteria used by the IMD to measure these variables. Performance on international test-score comparisons was not even mentioned among the 114 criteria used by the World Economic Forum.

Indeed, this should come as no surprise, because the PISA was never designed to be a predictor of economic competitiveness. Such tests may measure student knowledge and skills on an international basis, but they were not designed to measure the factors that contributed to student success within a nation. In 2013, in an international study of 1700 worldwide CEOs by IBM, the following four traits stand out as critical for an employee’s future success: Collaborative, Creative, Flexible, and Communicative. These four traits are not even measured on the PISA tests.

As a school superintendent for 16 years in Massachusetts, I have seen the value of state, national or international standardized tests when they provide practitioners with actionable data they can use to assess the overall effectiveness of their school or school district’s curriculum and instruction. Tests such as the PISA are useful tools if used for their intended purpose - to measure content knowledge and skills that students need to master for work and academic study after high school.

While it is true that the most recent U.S. scores were average among 62 education systems, the data do not tell us why one nation scored higher or lower than another. Unfortunately, U.S. Secretary of Education Arnie Duncan recently misused the student results and entirely blamed the public schools for the nation’s average PISA results. When the PISA results were released in 2013, Duncan stated:

PISA is an important, comparative snapshot of U.S. performance because the assessment is taken by 15 year-olds in high schools around the globe. The big picture of U.S. performance on the 2012 PISA is straightforward and stark: It is a picture of educational stagnation... educational complacency and low expectations.

Even the OECD authors of the PISA test acknowledge that PISA results are due to a combination of variables, including but not limited to schooling, life experiences/home environment, poverty, access to early childhood programs, and health. In 2013, the OECD wrote in one of their reports that poverty explains up to 46% of the PISA mathematics score in OECD countries. At no time did OECD claim, as Duncan stated, schools’ performance on the test can be blamed on low expectations and complacency.

While it may be the case that OECD has never claimed that PISA is a direct proxy for economic competitiveness, still, it’s come awfully close, providing politicians here and elsewhere with an irresistible opportunity to make that leap in logic. In releasing the 2013 PISA results, for example, OECD Secretary-General Angel Gurria linked schools and education to high levels of youth unemployment, rising inequality, and a pressing need to boost economic growth and national competitiveness. Not even a footnote pointed to the considerable contributions of Wall Street and the City of London to these very problems.

PISA and other international standardized tests are useful in some contexts and not in others. These tests are not either entirely predictive of everything, nor entirely useless. Let’s not overstate the usefulness of international standardized tests, but let’s not condemn them completely.

If U.S. education policy is really concerned about student success after high school and college, then I suggest our nation needs to establish effective policies that will reduce childhood poverty and will ensure all students a high quality public academic instruction. I also recommend K-12 educators listen to the 1,700 worldwide CEOs who need graduates who are collaborative, creative, flexible, and communicative.

Paul B. Ash, Lexington Public Schools, Massachusetts

The opinions expressed in Assessing the Assessments are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.