Opinion
Education Opinion

A Deeper Look at PISA Results

By Robert Rothman — February 28, 2014 3 min read
  • Save to favorites
  • Print

The results of the 2012 Programme for International Student Assessment (PISA), released in December, received a great deal of attention, with most news accounts focusing on the overall U.S. results. (In case you missed it, the release event and related information can be found here.) American fifteen-year-olds’ performance was flat over the past decade in reading, mathematics, and science, and the relative standing of U.S. students fell slightly because other nations leaped ahead.

But PISA provides a lot more information than just a scorecard of where nations perform in the international rankings. Significantly for this blog, PISA says a lot about deeper learning.

From its inception, PISA was designed to assess students’ ability to use knowledge to solve problems, many of which have real-world applications. In that respect, PISA is a better measure of deeper learning than many tests in use in the United States, which place a great deal of emphasis on factual recall, simple reading comprehension, and the ability to apply routine math procedures. The most challenging PISA items get at an even deeper level of knowledge and skill. Students who can do well on those, the ones who score at Level 5 and 6 on the assessment’s six-level performance scale, are indeed deep learners. For example, Level 5 in reading represents

[t]asks . . . that involve retrieving information require the reader to locate and organise several pieces of deeply embedded information, inferring which information in the text is relevant. Reflective tasks require critical evaluation or hypothesis, drawing on specialised knowledge. Both interpretative and reflective tasks require a full and detailed understanding of a text whose content or form is unfamiliar. For all aspects of reading, tasks at this level typically involve dealing with concepts that are contrary to expectations.

My organization, the Alliance for Excellent Education, conducted an analysis of the PISA results, focusing on performance at the top levels, to see which countries do well in enabling large numbers of students to demonstrate such abilities. We called our report “The Deepest Learners.”

As might be expected, countries that performed well overall also had a high proportion of students at the top levels. In Shanghai-China, 56 percent of students were top performers in at least one subject area, and 19.6 percent were top performers in all three. Among the industrialized nations in the Organisation for Economic Cooperation and Development (OECD), which administers PISA, on average 16.2 percent were top performers in at least one subject and 4.4 percent were top performers in all three. However, there were some variations among the countries. In Finland, for example, 24 percent were top performers in at least one subject and 7.4 percent were top performers in all three; in Japan, 30 percent were top performers in at least one subject and 11.3 percent in all three; and in Canada, 21.9 percent were top performers in at least one subject and 6.5 percent in all three. In the U.S., 12 percent of students were top performers in at least one subject and 4.7 percent were top performers in all three.

But in contrast to the overall results for the United States, which were flat, the proportion of American fifteen-year-olds performing at the top levels has declined over time, particularly in reading. In that subject, the proportion of top performers declined from 12.2 percent in 2000 to 7.9 percent in 2012. By contrast, in Poland, one of the surprises in the 2012 PISA results, the proportion of top performers in reading rose from 5.9 percent to 10 percent since 2000. (For more on Poland’s success, see Amanda Ripley’s excellent book, The Smartest Kids in the World.)

While no country, except Shanghai, has succeeded in bringing all or even most of its students to these high levels, these abilities are important. But clearly some do exceptionally well. And those that do appear to have built education systems that are designed to strengthen these abilities. For example, they have standards and curricula that ask students to engage in active inquiry, and use assessments that measure these abilities. Many also prepare teachers by using the same approaches they want students to demonstrate.

The United States is moving in this direction, with the adoption of the Common Core State Standards and new assessments. Perhaps the 2015 PISA results will tell a different story.

In the meantime, the OECD will soon release a report analyzing what the 2012 PISA results say about student problem-solving abilities. Look for more about that report on this blog.

The opinions expressed in Learning Deeply are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Building Teacher Capacity for Social-Emotional Learning
Set goals that support adult well-being and social-emotional learning: register today!


Content provided by Panorama
Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Tiny Wrists in Cuffs: How Police Use Force Against Children
An investigation finds children as young as 6 and a disproportionate amount of Black children have been handled forcibly by police officers.
15 min read
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Jhaimarion, 10, reacts as he listens to his mother, Krystal Archie talking with an Associated Press reporter in Chicago on Sept. 23, 2021. Archie’s three children were present when police, on two occasions, just 11 weeks apart, kicked open her front door and tore through their home searching for drug suspects. She’d never heard of the people they were hunting. Her oldest child, Savannah was 14 at the time; her youngest, Jhaimarion, was seven. They were ordered to get down on the floor.
Nam Y. Huh/AP
Education Gunman in 2018 Parkland School Massacre Pleads Guilty
A jury will decide whether Nikolas Cruz will be executed for one of the nation’s deadliest school shootings.
3 min read
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Annika Dworet and her husband, Mitch Dworet, wipe away tears as their son's name is read aloud during Marjory Stoneman Douglas High School shooter Nikolas Cruz's guilty plea on all 17 counts of premeditated murder and 17 counts of attempted murder in the 2018 shootings, at the Broward County Courthouse in Fort Lauderdale, Fla. on Wednesday, Oct. 20, 2021. The Dworet's son, Nicholas Dworet, 17, was killed in the massacre.
Amy Beth Bennett/South Florida Sun Sentinel via AP
Education Briefly Stated: October 20, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Gunman in Parkland School Massacre to Plead Guilty
The gunman who killed 14 students and three staff members at a Florida high school will plead guilty to their murders, his attorneys said.
4 min read
Parkland school shooter Nikolas Cruz is sworn in before pleading guilty, Friday, Oct. 15, 2021, at the Broward County Courthouse in Fort Lauderdale, Fla., on all four criminal counts stemming from his attack on a Broward County jail guard in November 2018, Cruz's lawyers said Friday that he plans to plead guilty to the 2018 massacre at a Parkland high school.
Parkland school shooter Nikolas Cruz is sworn in before pleading guilty, Friday, Oct. 15, 2021, at the Broward County Courthouse in Fort Lauderdale, Fla., on all four criminal counts stemming from his attack on a Broward County jail guard in November 2018, Cruz's lawyers said Friday that he plans to plead guilty to the 2018 massacre at a Parkland high school.
Amy Beth Bennett/South Florida Sun Sentinel via AP