To the Editor:
Leanna Stiefel and Amy Ellen Schwartz offer the very best reductionist, positivistic social science in their claim that they can measure some quantity in schooling called the “educational production function” (“Efficiency: The Missing Metric,” Commentary, Sept. 8, 2004).
Ms. Stiefel and Ms. Schwartz take one variable, dollars spent per student in a school, and they use regression analysis to identify how well each school performs on standardized tests, given the characteristics of its students that “make education more or less difficult” (this means the percentage of students eligible for free lunches and with limited English proficiency). So they claim that the poorer a school’s students are, the more who don’t speak English, the higher the school’s test scores with less money spent per student equals more efficient production.
This equation reduces the function of schooling entirely to test scores. It requires a “flatland” conception of human life, and it conveys a profound contempt for the phenomenological quality of human experience in schools. It also ignores just about all of what scientists learned in the 20th century about complex systems and living systems.
Forget about schools that foster resilience. Forget about the arts and athletics. Forget about relationships and caring. Forget about every single quality of life that matters for children and teenagers. The only thing that counts is test scores per dollar per factors “that make education more or less difficult.”
The Stiefel/Schwartz “educational production function” takes the bad policy of test obsession, ratchets up the intensity of obsession, and makes the whole thing obscene. Perhaps it’s just what we need to see how destructive our testing policies have become to the lives of our children.
David Marshak
Professor of Education
Seattle University
Seattle, Wash.
To the Editor:
We are really delighted that our work has caught David Marshak’s interest and hope that this is the beginning of a constructive, if spirited, debate. There are two things in his letter that we would like to clarify at the outset. First, please be aware that we have used measures beyond overall spending per pupil in our work—including spending measures disaggregated into categories such as instructional and noninstructional spending and other nonspending measures, such as pupil-teacher ratio, and characteristics of the teachers, such as percentage with master’s degrees. That said, we agree that it would be better to have more detail and nuance in the variables capturing inputs. Our choices were limited by the availability of data.
Second, we agree with Mr. Marshak that focusing exclusively on test scores and ignoring the other aspects of schooling is a mistake. Ideally, policymakers, parents, and society would place as much value on the arts, civics, and physical fitness, for example, as they currently seem to place on test scores.
Taking the heavy emphasis on test scores as a given, however, we argue that it is a mistake to look only at the scores and not account for differences in resources and students across schools. Too often schools are identified as “good” because their test scores are high and “bad” because they are low. Good schools are then rewarded or looked to as exemplars of best practice. Bad schools run the risk of sanctioning or re-organization. Unfortunately, this may have the effect of rewarding schools for attracting high-scoring students or turning away low-scoring students.
Is this the behavior we want other schools to model? We don’t think so. Further, this method may have the effect of celebrating schools for having the good fortune of small classes or highly skilled teachers. Is it fair or helpful to vilify low-scoring schools struggling with large classes and inexperienced teachers? Again, we don’t think so.
In the end, our production-function-based methodology attempts to control for these differences to disentangle the effectiveness of the schools, given the resources and students they have. Does it do so in a fully satisfying way? Probably not. Is it better than relying solely on test scores? We believe so. Using our limited resources well requires looking beyond scores and controlling for differences in circumstances, resources, and students, and, yes, looking at the other “products” of education: resilience, self-esteem, physical fitness, maturity, and more. We believe our production-function-based efficiency measure is a step in the right direction, even if it doesn’t take us to the finish line.
Amy Ellen Schwartz
Professor of Public Policy
Leanna Stiefel
Professor of Economics
New York University
New York, N.Y.
