To the Editor:
I read with considerable interest your article “‘No Effects’ Studies Raising Eyebrows” (April 1, 2009), which reported on a recent lack of findings from research sponsored by the Institute of Education Sciences. As befitting the story’s suggestive April Fools’ Day publication, it seemed to obscure the fact that any research design, including randomized controlled trials, has the potential to be poorly executed or misinterpreted in a way that arches eyebrows.
The relative prevalence of “no effects” findings in such trials, moreover, is hardly a compelling indictment on the increased use of this methodology. Rather, it is exactly what we should expect when undertaking credible assessments of interventions focused on the difficult challenges of helping students and teachers achieve their potential.
Nonetheless, there are still ways in which the institute’s research might be improved. Most notably, we could reasonably expect fewer “no effects” findings if the IES were to direct research agendas with the same political and budgetary autonomy that characterizes the National Science Foundation and the National Institutes of Health.
Analogies to medical research and practice can be particularly apt as the mix of methodologies used in education research continues its transition toward those used in other professional and scientific fields. We all want doctors and nurses who practice a craft that is wise, nuanced, responsive, and compassionate in ways that cannot easily be evaluated by randomized controlled trials. But we can and do rightfully insist that the training and practice of this craft also be informed by extensive bodies of rigorous evidence that only these trials can generate.
Thomas S. Dee
Associate Professor of Economics
Director, Public Policy Program
Swarthmore College
Swarthmore, Pa.
To the Editor:
Ah, the futility of it all. So much money, so much time spent, and so little to show for it.
Despite several valiant responses by supporters to the disappointing results of “scientifically based” education research studies, I have to question the apparently overly structured methodology applied to some messy education questions. Advocates quoted in your article say it’s “in the nature of evaluation science to find more inconclusive findings than positive findings, and that’s informative,” and others are expecting “more luck with the next cohort of studies.” Will evaluation science improve prior to the next cohort, or will future studies perhaps not be based on “fairly weakly supported ideas”?
We just cannot get over ourselves trying to make teaching and learning into a science. Teaching is an art for those born with the talent, and a craft for those inspired to acquire it. Learning is a life process that will never be captured conclusively for replication. But those of us who love learning will try and try again to impart a love for it in students whose innate desire to learn has been starved or poorly nourished.
We need to be careful what theses we attempt to prove, what causation we attempt to claim. We are arrogant in our overreaching for “truth,” and we forget what we have already learned and observed over time. I recommend that readers note Peter Berger’s Commentary “Predicting the Past,” in the same issue. Unlike our hapless researchers who have turned up with “no effects,” Mr. Berger reminds us that we already know them, yet we ignore much of our knowledge of teaching and learning. That is why we are, as he says, in the 21st century without mastering the skills of the 20th century and before.
Gillian B. Thorne
Executive Director, Office of Early College Programs
Director, UConn Early College Experience Program
University of Connecticut
Storrs, Conn.
