Opinion
International Opinion

International Test Scores, Irrelevant Policies

By Iris C. Rotberg — September 12, 2011 6 min read
  • Save to favorites
  • Print

Perhaps no research finding has influenced education policy more, or been subject to greater misinterpretation, than our ranking on international mathematics and science tests.

Previous critiques of international comparisons have focused largely on flaws in sampling and the limitations of test scores as a measure of the quality of a nation’s education system. These problems are still relevant. Equally important, however, are the conclusions drawn from the comparisons, even assuming their technical validity.

For decades, our rhetoric and education policies have been based on the premise that the ranking of U.S. students on international tests will lead to a decline in our nation’s economic competitiveness and a shortage of American scientists and engineers.

It is ironic, then, that given the rhetoric and policies surrounding international test-score comparisons—much of it unsupported by evidence—little attention is paid to two of the most powerful findings of these comparisons: the strong negative effects on student performance of both family poverty and concentrations of poverty in schools.

Instead, we draw conclusions from the international studies that are not supported either by the findings of these studies or by research more generally.

Poverty, not international test-score comparisons, is the most critical problem to be addressed by our public policies."

First, our rhetoric has assumed that test-score rankings are linked to a country’s economic competitiveness, yet the data for industrialized countries consistently show this assumption to be unwarranted. For example, the World Economic Forum’s 2010-2011 global-competitiveness report ranks the United States fourth, exceeded only by Switzerland, Sweden, and Singapore. Many of the countries that ranked high on test scores rank lower than the United States on competitiveness—for example, South Korea, No. 22, and Finland, No. 7.

Although we cannot predict future economic trends, we do know that test-score rankings are a poor basis upon which to understand these trends or to know what to do about them. The reason is clear: Other variables, such as outsourcing to gain access to lower-wage employees, the climate and incentives for innovation, tax rates, health-care and retirement costs, the extent of government subsidies or partnerships, protectionism, intellectual-property enforcement, natural resources, and exchange rates overwhelm mathematics and science scores in predicting economic competitiveness.

Second, we assume that U.S. students’ performance on math and science tests is reflected in a shortage of scientists, engineers, and mathematicians. The data, however, give a quite-different picture.

The fact is the United States has both a large pool of students with the academic credentials needed to enter science and engineering fields and an ample supply—and sometimes an oversupply (for example, of chemistry Ph.D.s)—to meet labor-market demand. That is the case even though slippage occurs between the number of graduates in science and engineering and the number who work in these fields, often because some graduates choose, for example, careers in finance, investment banking, management, or entrepreneurial activity. When companies claim that they need to hire from other countries because they cannot find qualified U.S. graduates, it is more likely that they cannot find them at the wages they would prefer to pay and find it cheaper to outsource. That is not the fault of our international test-score ranking or the training of U.S. scientists and engineers.

Moreover, U.S. Bureau of Labor Statistics projections show large variations in job opportunities among science and engineering fields. For example, employment in computer-software engineering; biological science; and biomedical, civil, and environmental engineering is expected to grow faster than the average for all occupations, while growth in computer programming; chemical and materials science; and electrical, mechanical, and marine engineering is expected to be slow. Although mathematics and physics are expected to have faster-than-average growth, the size of the market for those who seek basic-research positions is quite small.

Of the 30 occupations in the United States with the fastest rate of growth, only nine are in science and engineering fields, and 16 of the 30 do not require a college degree, according to Bureau of Labor Statistics projections. More important, of the 30 occupations expected to provide the largest numerical growth in jobs, only two (both in computer fields) are in science and engineering, and 23 do not require a college degree. If we consider only occupations requiring a college degree or above, 15 of the top 30 fastest-growing occupations are in science and engineering; however, only eight (six in computer fields) of the 30 occupations expected to provide the largest numerical growth in jobs are in science and engineering.

At the same time that our rhetoric has linked test scores, economic competitiveness, and shortages of scientists and engineers, our education policies have been dominated by test-based accountability, apparently with the expectation that accountability requirements would close the achievement gap, raise our ranking on international comparisons, and lead to a stronger economy and an increased supply of scientists and engineers. The assumption that accountability requirements are a solution to our education problems is as incongruous as our rhetoric about the economy and scientists and engineers.

BRIC ARCHIVE

Research accumulated over the years, analyzed in a 2011 National Research Council report, shows that accountability policies have not resulted in meaningful improvements in student learning and, in many instances, have created perverse incentives that weaken it. Yet, we continue to mandate accountability requirements that are not used—and in some cases are specifically discouraged—by the very countries whose test scores we most admire, including Finland and Japan.

At the same time, we have ignored the strongest evidence emerging from the international tests: the adverse effects of poverty and concentrations of poverty in schools on student performance in all countries.

Although countries can exacerbate or mitigate the impact of poverty through their social, fiscal, and education policies, and although some students do overcome the odds, the fact is the gap between high-poverty and more-affluent students remains a fundamental problem in virtually every country.

The 2009 Program for International Student Assessment, or PISA, findings for member-countries in the Organization for Economic Cooperation and Development show that, on average, close to 60 percent of the difference in reading performance between schools is accounted for by the socioeconomic status of the students attending the schools. In the United States, socioeconomic status accounts for close to 80 percent of the difference.

That gap is reflected throughout the students’ lives. It is specifically the low-income populations and regions that are underrepresented in mathematics, science, and engineering fields, and in professions generally—and it is these populations that are at the most severe disadvantage in competing for jobs in a global economy. This is part of a much broader set of problems faced by high-poverty populations. We have one of the largest divides between rich and poor in the industrialized world. One-fifth of our children live in poverty; millions of these children are concentrated in high-poverty schools—a setting that greatly compounds the problems of poverty.

Our policy deliberations work at the fringes of these realities, with remedies that are not focused on the basic problem of poverty. The problem will not be addressed by implementing tougher accountability requirements. Nor will it be addressed by rhetoric about mathematics and science scores, economic competitiveness, and generic shortages of scientists and engineers.

Poverty, not international test-score comparisons, is the most critical problem to be addressed by our public policies. Unfortunately, our recent political polarization over budgetary priorities does not leave much room for optimism.

A version of this article appeared in the September 14, 2011 edition of Education Week as International Test Scores, Irrelevant Policies

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

International What the Research Says It's Not Just U.S. Students. Civics Scores Have Dropped Around the World
Eighth graders are less engaged and knowledgeable about government than they were before the pandemic, a global study finds.
5 min read
vote ballot initiatives money 1371378601 01
LAUDISENO/iStock/Getty and EdWeek
International England Pushes for Cellphone Bans in Schools. Could the U.S. Be Next?
England is the latest country seeking to keep cellphones out of class.
3 min read
Tight crop photo of a student looking at their cellphone during class. The background is blurred, but shows students wearing uniforms.
E+
International Photos PHOTOS: Take a Round-the-World Tour of the Return to School
Here's what back to school looks like in classrooms around the globe.
1 min read
A teacher gives a lesson on the first day of school at a cadet lyceum in Kyiv, Ukraine on Sept. 4, 2023.
Young cadets sing the national anthem during a ceremony on the first day of school at a cadet lyceum in Kyiv, Ukraine, on Sept. 4, 2023.
Efrem Lukatsky/AP
International Opinion School Reform Is Tough All Over, Not Just in the U.S.
Even though some reforms produce evidence of student success, that often isn't enough to overcome political hurdles.
6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty