Assessment Opinion

Have You Seen This Headline?

By Carla Santorno — March 25, 2014 4 min read
  • Save to favorites
  • Print

Today’s guest contributor is Carla Santorno, Superintendent, Tacoma Public Schools, Washington.

Have you seen this headline?” “U.S. Teens Lag as China Soars on International Test.” (Hechinger, 2010) Or this? “Wake-up call: U.S. students trail global leaders."(Armario, 2010) Just as budding crocuses are the first sign of spring, these dire headlines announce the release of a new round of international tests (TIMSS or PISA). Lost in this welter of gloom-and-doom headlines is the reality only belatedly acknowledged by the test-makers: Somehow while assessing nearly 100% of the students in the United States and elsewhere in the West, they lost track of more than a quarter of the students assessed in Shanghai, almost all of them the children of low-income rural Chinese migrants (Stewart, 2014).

As a superintendent of a very diverse urban school district, I cannot control the media’s predictable assault on public education; however, I do have a responsibility to make meaning of these evaluative assessments in relation to our work in Tacoma. Where and what is the true signal in these international assessments from a cluster of noisy data?

Let me begin first with some background about my district. Tacoma is a researcher’s delight, a mid-sized urban school district, 57 schools, free and reduced lunch rates from 13 to 95%, we have some of the highest and lowest academically performing schools in the state. On the new WaKIDS screening tests some of our elementary schools have 98 percent of their students enter “kindergarten ready,” for others it is less than half. On-time graduation rates in our seven comprehensive high schools vary from 97 to 63 percent. Bilingual rates range from 1% to 40%. Some of our schools are 85 percent minority (defined as non-white) and others 20%.

With our rich diversity, we are a perfect microcosm to analyze the impact of demographics on academic performance. As most analysts now realize, the No Child Left Behind legislation has been problematic at many levels and its reauthorization a bipartisan disaster. However, the requirement to disaggregate data by subgroups has honed our skills in seeing how achievement is associated with ethnicity, poverty, English Language Learners, students with disabilities and other factors. And that is the exact perspective one needs to understand the international assessment results.

What does it mean to be first? Who is first? Why are they first? The answer can easily be captured in educational blogger and national principal of the year, Mel Riddile’s short tweet: “PISA: It’s Poverty Not Stupid”.

Perhaps the most effective whistle blower on the misleading interpretations of these global rankings was the late (and very much great) Gerald Bracey. When an international study of high school science and mathematics results were about to be released, Dr. Bracey noticed that Greece was substantially above the U.S. in both physics and math in high school. Having lived in Greece for a while, he faxed (pre-tweet days) “Are you kidding me?” He noted that 4th and 8th grade students in Greece performed near the bottom. He remarked to The Washington Post, “Do you really think these Greek kids suddenly encountered Socratic teachers in their high schools and shot their advanced students beyond ours? In a pig’s eye!”

Bracey and many that offer statistical analyses of these international rankings conclude that looking at results only from a competitive perspective does not tell the whole story. And the biggest devil in the details is called “poverty”. We have more social economic disparity than any other country. The difference between the “haves” and “haves not” in the U.S. is a gap unparalleled in the Western world. We are the most powerful and wealthy society in human history and yet somehow we have tolerated the discarding of a large portion of our children to lives of poverty. What happens to international test scores when we statistically account for the fact that Finland has a far lower percentage of students living in poverty (3%) compared to the US (20%)? (Irizarry, 2013) Looking at the data from a simple regression analysis that adjusted for family income would re-write the newspaper headline as follows: “Factoring for Poverty, U.S. Soars on International Tests”.

How does this issue translate in my district? One of my principals at a high-poverty high school recounted that he recently took 15 freshmen to an exhibit at the Pacific Science Center in Seattle, 40 minutes away, 30 miles. For 13 of the 15 students it was their first visit to Seattle.

That is what poverty looks like. That is the reality of many of our high school students, whose families want the best for them but lack resources. That is the equity and opportunity gap that should be a national wake-up call. And that is what we should be thinking about when reading the alarmist headlines generated by large-scale international assessments.

Carla Santorno, Tacoma Public Schools, Washington.

The opinions expressed in Assessing the Assessments are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.