skoolboy’s jaunt to the Olympics concludes today with an examination of how much going to one school versus another matters for students’ achievement in different countries. The basic approach is to look at the average achievement in a sample of schools within a country, and to see how much those averages differ from one another. If students were randomly distributed across schools in a country, and each school had similar resources, we might expect to see relatively similar average achievement across schools, and we might conclude that which school a student attends in that country doesn’t matter that much. On the other hand, if some schools in a country enroll poor students and others enroll wealthy students, and the schools serving poor students have fewer social, cultural, and economic resources available to support student achievement than the schools serving wealthy students, we might expect to see large differences in achievement across schools, suggesting that which school a student goes to in such a country matters a lot.
Data such as these don’t tell us about school effects , because they confound two different processes: selection into a school, and what happens to students after they enter the school. The latter is what we usually think of as a school effect. School-to-school differences in achievement could represent either selection or impact; they could occur because some schools raise students’ achievement more than others, or because schools enroll students who are already achieving at very different levels, or some combination of the two. In contrast, school-to-school differences in the social and economic composition of who is enrolled are best interpreted as evidence of selection, because going to one school or another doesn’t typically affect a student’s family background.
Once again, I’m using data from the PISA 2006 assessments of science, reading and math, a sample of about 30 OECD countries and an additional 25 partner countries or economies. (For those playing along at home, the data are from Chapter 4 of the report PISA 2006: Science Competencies for Tomorrow’s World.)
The first figure below shows the proportion of the variation in individual student achievement in a country that is between schools; put differently, how much the average achievement in a school differs from one school to the next within a country. I’ve averaged the proportions for reading, math and science for each country (they’re very highly correlated with one another.) This proportion can vary from 0% to 100%. Zero percent of the variance in achievement between schools would be observed if every school in the country had exactly the same average achievement, with some students in each school doing very well, and others doing poorly. It’s hard to picture what 100% of the variance between schools would look like, but imagine a ladder with many, many rungs that are pretty far apart, and each rung represents a particular school’s average achievement, with everybody in that school scoring right at the level of the rung. Some schools would have very high average achievement, and some would have very low average achievement, and there’d be no overlap among the schools—if you knew which school a student attended, you could predict that student’s performance perfectly.
Not surprisingly, the reality lies somewhere in between, and the figure shows that countries differ substantially from one another in how spread out achievement is across different schools. Fifteen countries, headed by Hungary, Slovenia, and Germany, have systems in which more than 50% of the variance in student achievement lies between schools. Conversely, Scandinavian countries have the most even distribution of student achievement across schools, headed by Finland, Iceland and Norway. In the U.S., 25% of the achievement of 15-year-olds is between schools, which is significantly lower than the proportion in 37 countries, and significantly higher than the proportion in a dozen countries.
The second figure shows the proportion of the variation in individual students’ socioeconomic background that is between schools—how much the school average socioeconomic status differs from one school to the next within a country, using the PISA index of economic, social and cultural status I described last week. If none of the variance in students’ socioeconomic status were between schools, we could say that students are randomly distributed across schools according to their socioeconomic backgrounds. If a great deal of the variance in students’ socioeconomic status is between schools, schools in that country are socially segregated from one another.
The U.S. is pretty much in the middle of the distribution of countries in terms of how spread out schools are from one another in their socioeconomic composition. 26% of the variance in individual student socioeconomic status is between schools in the U.S., which is significantly lower than 18 countries, and significantly higher than 16 countries. The countries that have the most socially segregated schools are headed by Chile, Bulgaria, Thailand, and Hungary; those that have the least socially segregated schools are the Scandinavian countries of Finland, Norway, Sweden, Denmark and Iceland.
It’s likely no surprise to thoughtful readers that schools differ substantially in their achievement levels and social compositions in most countries, but what is intriguing is that this happens in spite of the fact that there are substantial differences across countries in how education systems are organized, with some systems centralized, and others decentralized; variability in the extent to which schools are run by the state or by private entities such as religious institutions; and differences in the extent to which the secondary schools in a country prepare students for particular vocational or postsecondary destinations. The U.S. is recognized as a large, decentralized system of schools that are mostly local. Residential segregation by race, ethnicity and economic status leads to neighborhood schools that are similarly segregated, as poor people live in different places than rich folks, and therefore generally attend different schools. Increasingly, we see in the U.S. more explicit processes by which students and schools mutually select one another, on the basis of economic status (in the case of private schools charging tuition, or high-spending suburban districts with high property taxes), or on the basis of prior academic achievement (in the case of schools with entrance exams or, as eduwonkette has shown repeatedly in New York City, in the ways that new small high schools enroll higher-performing students than the large, comprehensive high schools they’ve replaced). It is important to recognize that when a school is selecting on achievement, it’s also selecting on social class background, and vice versa, because achievement and family background are correlated.
A final caveat: The PISA data I’ve reported are at the country level, but this may not be the most meaningful geographic unit when it comes to the distribution of students across schools by socioeconomic background and achievement. What we see at the national level might not apply to geographic subunits such as states, counties, or large school districts.