One of the most serious obstacles to a constructive discussion of the issues raised by our study, which was cited in your Nov. 10, 2004, article “NCLB Could Alter Science Teaching,” is the loose mapping between the actual study and the terms used to describe the findings or their implications.
For example, Alfie Kohn and Sharon Janulaw fear that our findings may be used to “return to a traditional, fact-oriented, teacher-centered model” (“Standardized Science,” Letters, Dec. 1, 2004). But in our study, the instructional objective was neither traditional nor fact-oriented. Instead, it was to teach children how to design and interpret unconfounded experiments, that is, how to vary only one factor at a time, how to avoid varying multiple factors simultaneously, and why it is possible to make an unambiguous inference from the former type of experiment but not the latter.
Mr. Kohn and Ms. Janulaw note that our study had limitations because we studied only “100 3rd and 4th graders.” While this is a relatively large sample size for the kind of careful experiment we were conducting, they are quite correct: This is just one study—although my colleagues and I have conducted several similar studies on several other classroom contexts, with similar findings. Therefore, our paper ends with a plea for more research—rather than rhetorical arguments—the aim of which, we note, “would be to generate an empirically sound basis for determining the most effective matches between topic, student, and type of pedagogy.”
“Such results,” we continue, “could provide evidence-based guidance to teachers for achieving a balanced portfolio of instructional approaches to early science instruction.”
Mr. Kohn and Ms. Janulaw also argue that the procedure followed in what we called the “discovery learning” condition is not representative of what is really recommended by discovery-learning advocates. To that critique, we have two replies. First, is it really so different? Our discovery condition presented the experimental apparatus to the children. It presented them with a goal, “see if you can set up the ramps to see if the height of the ramp makes a difference,” and then students were free to explore, in a hands-on fashion, various kinds of arrangements, run the experiments, observe the results, and finally, under teacher suggestion, move on to another goal, such as “see if you can set up the ramps to see if the surface of the ramp makes a difference in how far the ball rolls.”
I would venture that this is not so far from what passes for discovery learning in many elementary school classrooms. That said, it is important to recall that the main goal of our study was not to show that one form of instruction was more effective than another (we had already demonstrated that in several other studies). Instead, the goal was to show that once students master a procedure (such as how to design a simple, unconfounded experiment), then the way that they achieved that mastery—via one instructional method or another—doesn’t matter on a “far-transfer task.” The far-transfer task we used—children’s assessments of other children’s science-fair posters—would seem to be exactly the kind of “meaningful form of assessment” that Mr. Kohn and Ms. Janulaw argue for. All children who achieved mastery on the experimental design task were equally adept at making such assessments, regardless of how they had achieved mastery. This “path independence” hypothesis is the main point of the study co-authored by Milena Nigam and me.
Erin Marie Furtak’s letter, also in your Dec. 1, 2004, issue, is based on a careful reading of the original journal article, rather than on the way it was summarized in Education Week, and she reiterates our argument for the need for our field to make more precise use of terminology before moving on to policy decisions. Indeed, it is surprising that science educators so often abandon one of the foundations of science—the operational definition—when they engage in heated debates about discovery, inquiry, hands-on, and the rest. No science can advance without clear, unambiguous, operationally defined procedures. Neither can education science.
Finally, as a “bench scientist” who decided only in the past dozen years or so to venture out of the psychology lab into the messy and challenging world of educational research, I must say that I tremble when reporters call. Even with repeated iterations on exactly what my studies do and don’t suggest, I am often surprised to see the context in which they are presented (as in the provocative headline for the article that generated this exchange).
David Klahr
Professor of Psychology
Carnegie Mellon University
Pittsburgh, Pa.
