Opinion
International Commentary

International Test Scores, Irrelevant Policies

By Iris C. Rotberg — September 12, 2011 6 min read

Perhaps no research finding has influenced education policy more, or been subject to greater misinterpretation, than our ranking on international mathematics and science tests.

Previous critiques of international comparisons have focused largely on flaws in sampling and the limitations of test scores as a measure of the quality of a nation’s education system. These problems are still relevant. Equally important, however, are the conclusions drawn from the comparisons, even assuming their technical validity.

For decades, our rhetoric and education policies have been based on the premise that the ranking of U.S. students on international tests will lead to a decline in our nation’s economic competitiveness and a shortage of American scientists and engineers.

It is ironic, then, that given the rhetoric and policies surrounding international test-score comparisons—much of it unsupported by evidence—little attention is paid to two of the most powerful findings of these comparisons: the strong negative effects on student performance of both family poverty and concentrations of poverty in schools.

Instead, we draw conclusions from the international studies that are not supported either by the findings of these studies or by research more generally.

Poverty, not international test-score comparisons, is the most critical problem to be addressed by our public policies."

First, our rhetoric has assumed that test-score rankings are linked to a country’s economic competitiveness, yet the data for industrialized countries consistently show this assumption to be unwarranted. For example, the World Economic Forum’s 2010-2011 global-competitiveness report ranks the United States fourth, exceeded only by Switzerland, Sweden, and Singapore. Many of the countries that ranked high on test scores rank lower than the United States on competitiveness—for example, South Korea, No. 22, and Finland, No. 7.

Although we cannot predict future economic trends, we do know that test-score rankings are a poor basis upon which to understand these trends or to know what to do about them. The reason is clear: Other variables, such as outsourcing to gain access to lower-wage employees, the climate and incentives for innovation, tax rates, health-care and retirement costs, the extent of government subsidies or partnerships, protectionism, intellectual-property enforcement, natural resources, and exchange rates overwhelm mathematics and science scores in predicting economic competitiveness.

Second, we assume that U.S. students’ performance on math and science tests is reflected in a shortage of scientists, engineers, and mathematicians. The data, however, give a quite-different picture.

The fact is the United States has both a large pool of students with the academic credentials needed to enter science and engineering fields and an ample supply—and sometimes an oversupply (for example, of chemistry Ph.D.s)—to meet labor-market demand. That is the case even though slippage occurs between the number of graduates in science and engineering and the number who work in these fields, often because some graduates choose, for example, careers in finance, investment banking, management, or entrepreneurial activity. When companies claim that they need to hire from other countries because they cannot find qualified U.S. graduates, it is more likely that they cannot find them at the wages they would prefer to pay and find it cheaper to outsource. That is not the fault of our international test-score ranking or the training of U.S. scientists and engineers.

Moreover, U.S. Bureau of Labor Statistics projections show large variations in job opportunities among science and engineering fields. For example, employment in computer-software engineering; biological science; and biomedical, civil, and environmental engineering is expected to grow faster than the average for all occupations, while growth in computer programming; chemical and materials science; and electrical, mechanical, and marine engineering is expected to be slow. Although mathematics and physics are expected to have faster-than-average growth, the size of the market for those who seek basic-research positions is quite small.

Of the 30 occupations in the United States with the fastest rate of growth, only nine are in science and engineering fields, and 16 of the 30 do not require a college degree, according to Bureau of Labor Statistics projections. More important, of the 30 occupations expected to provide the largest numerical growth in jobs, only two (both in computer fields) are in science and engineering, and 23 do not require a college degree. If we consider only occupations requiring a college degree or above, 15 of the top 30 fastest-growing occupations are in science and engineering; however, only eight (six in computer fields) of the 30 occupations expected to provide the largest numerical growth in jobs are in science and engineering.

At the same time that our rhetoric has linked test scores, economic competitiveness, and shortages of scientists and engineers, our education policies have been dominated by test-based accountability, apparently with the expectation that accountability requirements would close the achievement gap, raise our ranking on international comparisons, and lead to a stronger economy and an increased supply of scientists and engineers. The assumption that accountability requirements are a solution to our education problems is as incongruous as our rhetoric about the economy and scientists and engineers.

03CopyrightedImage rotberg

Research accumulated over the years, analyzed in a 2011 National Research Council report, shows that accountability policies have not resulted in meaningful improvements in student learning and, in many instances, have created perverse incentives that weaken it. Yet, we continue to mandate accountability requirements that are not used—and in some cases are specifically discouraged—by the very countries whose test scores we most admire, including Finland and Japan.

At the same time, we have ignored the strongest evidence emerging from the international tests: the adverse effects of poverty and concentrations of poverty in schools on student performance in all countries.

Although countries can exacerbate or mitigate the impact of poverty through their social, fiscal, and education policies, and although some students do overcome the odds, the fact is the gap between high-poverty and more-affluent students remains a fundamental problem in virtually every country.

The 2009 Program for International Student Assessment, or PISA, findings for member-countries in the Organization for Economic Cooperation and Development show that, on average, close to 60 percent of the difference in reading performance between schools is accounted for by the socioeconomic status of the students attending the schools. In the United States, socioeconomic status accounts for close to 80 percent of the difference.

That gap is reflected throughout the students’ lives. It is specifically the low-income populations and regions that are underrepresented in mathematics, science, and engineering fields, and in professions generally—and it is these populations that are at the most severe disadvantage in competing for jobs in a global economy. This is part of a much broader set of problems faced by high-poverty populations. We have one of the largest divides between rich and poor in the industrialized world. One-fifth of our children live in poverty; millions of these children are concentrated in high-poverty schools—a setting that greatly compounds the problems of poverty.

Our policy deliberations work at the fringes of these realities, with remedies that are not focused on the basic problem of poverty. The problem will not be addressed by implementing tougher accountability requirements. Nor will it be addressed by rhetoric about mathematics and science scores, economic competitiveness, and generic shortages of scientists and engineers.

Poverty, not international test-score comparisons, is the most critical problem to be addressed by our public policies. Unfortunately, our recent political polarization over budgetary priorities does not leave much room for optimism.

A version of this article appeared in the September 14, 2011 edition of Education Week as International Test Scores, Irrelevant Policies

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Recruitment & Retention Webinar
Recruiting and Retaining a More Diverse Teaching Workforce
We discuss the importance of workforce diversity and learn strategies to recruit and retain teachers from diverse backgrounds.
Content provided by EdWeek Top School Jobs
Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children

EdWeek Top School Jobs

Principal
Meredith, New Hampshire
Inter-Lakes School District
Elementary Principal
Washington State
Wenatchee School District
Principal
Meredith, New Hampshire
Inter-Lakes School District
Elementary Principal
Washington State
Wenatchee School District

Read Next

International Global Test Finds Digital Divide Reflected in Math, Science Scores
New data from the 2019 Trends in International Math and Science Study show teachers and students lack digital access and support.
3 min read
Image of data.
iStock/Getty
International Pre-COVID Learning Inequities Were Already Large Around the World
A new international benchmarking highlights gaps in training for digital learning and other supports that could deepen the challenge for low-income schools during the pandemic.
4 min read
International Part of Global Trend, 1 in 3 U.S. High Schoolers Felt Disconnected From School Before Pandemic
UNESCO's annual report on global education progress finds countries need to make more effort to include marginalized students, particularly in the United States.
4 min read
International How Schools in Other Countries Have Reopened
Ideas from Australia, Denmark, and Taiwan can help American district and school leaders as they shape their reopening plans.
11 min read
Students at the Taipei American School in Taipei, Taiwan, perform The Little Mermaid in full costume and masks.
Students at the Taipei American School in Taipei, Taiwan, perform The Little Mermaid in full costume and masks.
Photo courtesy of Dustin Rhoades/Taipei American School