What do international comparisons really tell us about how American education can improve?
As global-benchmarking studies—such as the Program for International Student Assessment, the Trends in International Mathematics and Science Study, and others—gain a higher profile in policy discussions, a new study suggests there’s still much to be learned about how social and economic factors shape education outcomes worldwide.
“The Iceberg Effect,” released last week by the Horace Mann League of the U.S.A. and the National Superintendents Roundtable, a Seattle-based group of district leaders in 30 states, aims to put student achievement and long-term education outcomes within the broader social and economic contexts of different countries.
James J. Harvey, the executive director at the National Superintendents Roundtable and the author of the study, argued in a briefing on the study in Washington that short-term student outcomes, such as results from international reading or math tests, and even system outcomes, like the average education level of adults, are only the “tip of the iceberg” in getting a read on an education system’s overall health.
In a new international study, U.S. schools rank poorly for social stress, such as having high rates of births or violent deaths among young people, but the nation’s education system still gets top marks for long-term outcomes like having a high proportion of educated adults.
Source: “The Iceberg Effect,” Horace Mann League of the U.S.A., National Superintendents
“When we cherry-pick one piece of information, like PISA scores, we are acting as though that’s the only thing that counts,” Mr. Harvey said in an interview on the study. “We wanted to get a discussion started about the fact that, for 30 years, we’ve been fiddling around with the tip of the iceberg and done nothing at all to address all this that’s going on below the water line.”
The study came in the same week that the Organization for Economic Cooperation and Development, the Paris-based group that administers PISA, released reports finding that most countries have not evaluated or followed up on education policies launched in the last decade. They also showed that across the 37 countries studied, 17 percent of young people, on average, leave education without a secondary-level credential, such as a high school diploma.
The Invisible ‘Iceberg’
So what’s going on below the surface?
For his report, Mr. Harvey winnowed data from the last five years in 40 international-data systems for nine industrialized countries considered economically and educationally similar: Canada, China, Finland, France, Germany, Italy, Japan, the United Kingdom, and the United States. The report compares the countries on 24 indicators in six areas, with a total of 40 points available in each area.
In addition to student and system outcomes, the study identified four “dimensions” that may contribute to students’ health and educational growth:
• Economic equity, including income inequality, child poverty, infant-mortality rates, and economic mobility across generations;
• Social stress, including rates of violent and drug-related deaths overall, births to parents who are ages 15 to 19, and the percentage of each country’s population that is foreign-born;
• Family supports, including access to preschool, child-death rates as a result of abuse or neglect, and family benefits—both as total spending on families with two children and as a percentage of a nation’s gross domestic product; and
• School supports, including total spending and spending as a percentage of each nation’s GDP, average class size, and teacher workload.
The data show dramatic differences in the educational contexts of different countries, but no clear patterns arise about how different social and economic supports or stresses play into student achievement in either the short or long term.
The United States, for example, has the best long-term system outcomes of any country, with the most-educated adults and a significant share of the world’s high achievers in science on the PISA. But it has middling short-term student outcomes and provides nearly the lowest level of support for families, even in the face of high levels of economic inequality and social stress. (See chart, this page.)
The United Kingdom provides better supports and has lower levels of social stress and inequality, but has poorer student achievement and adult education than the United States.
The data point to dramatic differences in the scale and demographics of the countries to which the United States often compares itself.
“As a school leader, I’ve been frustrated every time PISA comes out,” said Janet M. Robinson, the superintendent of the Stratford, Conn., public schools and a roundtable member, at the briefing. “People take as common wisdom that our schools are totally inadequate compared to these others,” she said, but the study suggests it’s difficult to tell whether American schools would benefit from adopting different countries’ policies and practices.
No Clear Patterns
For example, the study found that while students take several benchmark exams at age 15, that age represents a different point in students’ educational careers in different countries.
In the United States, 15-year-olds are usually in 9th or 10th grade, with a few more years until graduation. In several European countries, that point marks the end of secondary school. In China, there is a major gateway exam between 9th and 10th grades, when most students take international assessments.
“Once you start trying to dig into a country’s system—their methods of generating data and their approach to separate populations in the country—you recognize it is apples and oranges and pears,” said Charles Fowler, the executive director of the online Suburban School Superintendents and the president-elect of the Port-Ludlow, Wash.-based Horace Mann League, a nonprofit advocacy and research group dedicated to public education.
Finland, for example, has fewer than 543,000 students in its comprehensive system, which serves students ages 7 to 15. That’s about 100,000 fewer students than the Los Angeles public school district, according to the district and Statistics Finland.
To take that comparison a step further, “The Iceberg Effect” notes that a little more than 5 percent of children 17 and younger in Finland live in families earning less than half the country’s median income, a marker of poverty. More than 80 percent of Los Angeles public school students and more than half of U.S. public school students nationwide are considered low income.
According to the report, Finland has greater equity, fewer social stresses, and better supports than nearly every other country in the study, and better short-term outcomes for its students, but it ranks below the United States, Canada, and the United Kingdom in long-term education outcomes.
Similarly, China has no data available on its supports for schools and young families, or on student performance in the vast majority of the country. Shanghai, often used as a stand-in for China as a whole, represents only about 10 percent of the nation’s students, and none with disabilities.
“Our hope is by putting these findings out in front of people, anomalous as some may be, it will stimulate research to get at some of these strange results,” Mr. Fowler said. “You can have all the resources in the world, but if you cannot bring to bear the kind of social and health resources needed to improve the mental and economic well-being of children and families, you are not going to make much progress.”
