Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
School & District Management Letter to the Editor

‘No Effects’ Research and the Realities of Learning

April 21, 2009 3 min read

To the Editor:

I read with considerable interest your article “‘No Effects’ Studies Raising Eyebrows” (April 1, 2009), which reported on a recent lack of findings from research sponsored by the Institute of Education Sciences. As befitting the story’s suggestive April Fools’ Day publication, it seemed to obscure the fact that any research design, including randomized controlled trials, has the potential to be poorly executed or misinterpreted in a way that arches eyebrows.

The relative prevalence of “no effects” findings in such trials, moreover, is hardly a compelling indictment on the increased use of this methodology. Rather, it is exactly what we should expect when undertaking credible assessments of interventions focused on the difficult challenges of helping students and teachers achieve their potential.

Nonetheless, there are still ways in which the institute’s research might be improved. Most notably, we could reasonably expect fewer “no effects” findings if the IES were to direct research agendas with the same political and budgetary autonomy that characterizes the National Science Foundation and the National Institutes of Health.

Analogies to medical research and practice can be particularly apt as the mix of methodologies used in education research continues its transition toward those used in other professional and scientific fields. We all want doctors and nurses who practice a craft that is wise, nuanced, responsive, and compassionate in ways that cannot easily be evaluated by randomized controlled trials. But we can and do rightfully insist that the training and practice of this craft also be informed by extensive bodies of rigorous evidence that only these trials can generate.

Thomas S. Dee

Associate Professor of Economics

Director, Public Policy Program

Swarthmore College

Swarthmore, Pa.

To the Editor:

Ah, the futility of it all. So much money, so much time spent, and so little to show for it.

Despite several valiant responses by supporters to the disappointing results of “scientifically based” education research studies, I have to question the apparently overly structured methodology applied to some messy education questions. Advocates quoted in your article say it’s “in the nature of evaluation science to find more inconclusive findings than positive findings, and that’s informative,” and others are expecting “more luck with the next cohort of studies.” Will evaluation science improve prior to the next cohort, or will future studies perhaps not be based on “fairly weakly supported ideas”?

We just cannot get over ourselves trying to make teaching and learning into a science. Teaching is an art for those born with the talent, and a craft for those inspired to acquire it. Learning is a life process that will never be captured conclusively for replication. But those of us who love learning will try and try again to impart a love for it in students whose innate desire to learn has been starved or poorly nourished.

We need to be careful what theses we attempt to prove, what causation we attempt to claim. We are arrogant in our overreaching for “truth,” and we forget what we have already learned and observed over time. I recommend that readers note Peter Berger’s Commentary “Predicting the Past,” in the same issue. Unlike our hapless researchers who have turned up with “no effects,” Mr. Berger reminds us that we already know them, yet we ignore much of our knowledge of teaching and learning. That is why we are, as he says, in the 21st century without mastering the skills of the 20th century and before.

Gillian B. Thorne

Executive Director, Office of Early College Programs

Director, UConn Early College Experience Program

University of Connecticut

Storrs, Conn.

Related Tags:

A version of this article appeared in the April 22, 2009 edition of Education Week as ‘No Effects’ Research and The Realities of Learning


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Learn directly from the pros why K-12 branding and marketing matters, and how to do it effectively.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Join experts from Samsung and Boxlight to learn how to make learning more interactive from anywhere.
Content provided by Samsung
Teaching Live Online Discussion A Seat at the Table With Education Week: How Educators Can Respond to a Post-Truth Era
How do educators break through the noise of disinformation to teach lessons grounded in objective truth? Join to find out.

EdWeek Top School Jobs

BASE Program Site Director
Thornton, CO, US
Adams 12 Five Star Schools
Director of Information Technology
Montpelier, Vermont
Washington Central UUSD
Great Oaks AmeriCorps Fellow August 2021 - June 2022
New York City, New York (US)
Great Oaks Charter Schools
Director of Athletics
Farmington, Connecticut
Farmington Public Schools

Read Next

School & District Management Student Mental Health and Learning Loss Continue to Worry Principals
Months into the pandemic, elementary principals say they still want training in crucial areas to help students who are struggling.
3 min read
Student sitting alone with empty chairs around her.
Maria Casinos/iStock/Getty
School & District Management Opinion A Road Map for Education Research in a Crisis
Here are five basic principles for a responsible and timely research agenda during the COVID-19 pandemic.
Robin J. Lake
4 min read
Two opposing sides reaching out to work together
J.R. Bee for Education Week
School & District Management 1,000 Students, No Social Distancing, and a Fight to Keep the Virus Out
A principal describes the "nightmare" job of keeping more than 1,000 people safe in the fast-moving pandemic.
4 min read
Dixie Rae Garrison, principal of West Jordan Middle School, in West Jordan, Utah.
Dixie Rae Garrison, principal of West Jordan Middle School in West Jordan, Utah, would have preferred a hybrid schedule and other social distancing measures.
Courtesy of Dixie Rae Garrison
School & District Management A School Leader Who Calls Her Own Shots on Battling the Coronavirus
A charter school founder uses her autonomy to move swiftly on everything from classroom shutdowns to remote schooling.
3 min read
Nigena Livingston, founder and head of School at the URBAN ACT Academy in Indianapolis, Ind.
Nigena Livingston, founder and head of school at the URBAN ACT Academy in Indianapolis, makes swift decisions in responding to the threat of COVID-19 in her school community.
Courtesy of Nigena Livingston