At a time when states and districts are on the lookout for textbooks and related instructional materials to guide the teaching of the common core, a recent study offers a cautionary tale about the selection of publishers’ wares.
First, don’t underestimate the importance to student learning of the particular curricular program chosen. Using different curricula, even if they share the same general pedagogical approach, can produce significant differences in learning gains, the study finds. Second, a solid research base is lacking on the effectiveness of competing programs, depriving schools and districts of critical information to help inform their choices. The good news, they suggest, is that this problem would be “relatively inexpensive” to remedy. (Hat tip to the National Council of Teacher Quality blog for bringing this study to our attention.)
The study is not brand new, as it was published in May by the journal Educational Evaluation and Policy Analysis. You can find a copy of the full study here.
The authors, Rachana Bhatt of Georgia State University and Cory Koedel of the University of Missouri, examined the three most popular elementary math programs used in Indiana between 1998 and 2004: Saxon Math, Silver-Burdett Ginn Mathematics, and Scott Foresman-Addison Wesley Math. Indiana was selected, the authors say, because it’s one of just two states they could find that collect and make publicly available data on which curricular materials local districts adopt.
Of the three publishers’ programs examined, the one identified as being most effective as measured by improvements over time in state test scores, Silver-Burdett Ginn, is no longer offered in Indiana. And the publisher of the curriculum found to be least effective in the study, (Saxon) retained a strong market share in the following adoption cycle.
(Saxon Publishers has sponsored some evaluations of its programs, including a 2005 review of “independent research” conducted by a team from the University of Oklahoma, that identifies the “positive impact” that Saxon Math has had on students’ math achievement. Meanwhile, a 2006 study funded by the publisher found that students using Saxon math in grades K-3 made “significant achievement gains” on a standardized test.)
The new study suggests that the issue of the instructional strategy behind different curricular programs may get more attention than is due. “With researchers and policymakers placing so much emphasis on differences between the traditional and reform pedagogies, our findings serve as a reminder that other differences should not be overlooked,” the study says.
The researchers voice particular concern that districts and states typically have very little data to draw on when making decisions on a new curriculum.
"[T]he research literature in this area is so thin,” they write, suggesting that a series of “independent evaluations from multiple contexts, taken together, could provide valuable information about the effectiveness of various curricular alternatives.”
A key problem, the study says, is that most state education agencies simply do not provide information about which curricular programs are being used in schools and districts. In fact, many don’t collect this information at all.
“Such data would be cheap and easy to collect, particularly compared to other data elements in many state longitudinal systems, and could be used to learn much about this important educational resource,” the study says.
Indeed, the study signals that the data would also help educators make more nuanced choices about finding the right materials for the right situation.
Concern about a lack of research on existing curricular materials was also raised in a recent report from the Brookings Institution. In fact, that report also included a call for all states to start collecting data on the textbooks in use.
“Not only is little information available on the effectiveness of most instructional materials, there is also very little systematic information on which materials are being used in which schools,” the Brookings report says. “This scandalous lack of information will only become more troubling” as districts begin to implement the Common Core State Standards, as well as efforts to improve teacher effectiveness.
These issues call to mind a book I blogged about earlier this year, The Tyranny of the Textbook. Written by Beverlee Jobrack, a veteran educational publisher, the book makes the case that curriculum is often a neglected ingredient in efforts to improve education. So in that regard, there may well be common ground with the folks at Brookings and the authors of the new study. That said, Jobrack’s analysis would seem to suggest that a better research base may not drive better decisions.
She describes a textbook development and selection system that is “perpetuating mediocrity.” And she says there’s plenty of blame to go around, including a “radically consolidated” publishing industry “driven by sales and marketing teams"; school and district committees for selecting curriculum filled with teachers and others who typically lack the appropriate time, expertise, and motivation to make wise choices; and state textbook adoptions focused on adherence to standards with little or no attention to whether the materials reflect a coherent and well-designed instructional approach.
In other words, if a robust research base existed, would it be put to good use?