Study Finds Achievement Differences Tied to Curricular Choices

By Erik W. Robelen — September 20, 2012 4 min read
  • Save to favorites
  • Print

At a time when states and districts are on the lookout for textbooks and related instructional materials to guide the teaching of the common core, a recent study offers a cautionary tale about the selection of publishers’ wares.

First, don’t underestimate the importance to student learning of the particular curricular program chosen. Using different curricula, even if they share the same general pedagogical approach, can produce significant differences in learning gains, the study finds. Second, a solid research base is lacking on the effectiveness of competing programs, depriving schools and districts of critical information to help inform their choices. The good news, they suggest, is that this problem would be “relatively inexpensive” to remedy. (Hat tip to the National Council of Teacher Quality blog for bringing this study to our attention.)

The study is not brand new, as it was published in May by the journal Educational Evaluation and Policy Analysis. You can find a copy of the full study here.

The authors, Rachana Bhatt of Georgia State University and Cory Koedel of the University of Missouri, examined the three most popular elementary math programs used in Indiana between 1998 and 2004: Saxon Math, Silver-Burdett Ginn Mathematics, and Scott Foresman-Addison Wesley Math. Indiana was selected, the authors say, because it’s one of just two states they could find that collect and make publicly available data on which curricular materials local districts adopt.

Of the three publishers’ programs examined, the one identified as being most effective as measured by improvements over time in state test scores, Silver-Burdett Ginn, is no longer offered in Indiana. And the publisher of the curriculum found to be least effective in the study, (Saxon) retained a strong market share in the following adoption cycle.

(Saxon Publishers has sponsored some evaluations of its programs, including a 2005 review of “independent research” conducted by a team from the University of Oklahoma, that identifies the “positive impact” that Saxon Math has had on students’ math achievement. Meanwhile, a 2006 study funded by the publisher found that students using Saxon math in grades K-3 made “significant achievement gains” on a standardized test.)

The new study suggests that the issue of the instructional strategy behind different curricular programs may get more attention than is due. “With researchers and policymakers placing so much emphasis on differences between the traditional and reform pedagogies, our findings serve as a reminder that other differences should not be overlooked,” the study says.

The researchers voice particular concern that districts and states typically have very little data to draw on when making decisions on a new curriculum.

"[T]he research literature in this area is so thin,” they write, suggesting that a series of “independent evaluations from multiple contexts, taken together, could provide valuable information about the effectiveness of various curricular alternatives.”

A key problem, the study says, is that most state education agencies simply do not provide information about which curricular programs are being used in schools and districts. In fact, many don’t collect this information at all.

“Such data would be cheap and easy to collect, particularly compared to other data elements in many state longitudinal systems, and could be used to learn much about this important educational resource,” the study says.

Indeed, the study signals that the data would also help educators make more nuanced choices about finding the right materials for the right situation.

Concern about a lack of research on existing curricular materials was also raised in a recent report from the Brookings Institution. In fact, that report also included a call for all states to start collecting data on the textbooks in use.

“Not only is little information available on the effectiveness of most instructional materials, there is also very little systematic information on which materials are being used in which schools,” the Brookings report says. “This scandalous lack of information will only become more troubling” as districts begin to implement the Common Core State Standards, as well as efforts to improve teacher effectiveness.

These issues call to mind a book I blogged about earlier this year, The Tyranny of the Textbook. Written by Beverlee Jobrack, a veteran educational publisher, the book makes the case that curriculum is often a neglected ingredient in efforts to improve education. So in that regard, there may well be common ground with the folks at Brookings and the authors of the new study. That said, Jobrack’s analysis would seem to suggest that a better research base may not drive better decisions.

She describes a textbook development and selection system that is “perpetuating mediocrity.” And she says there’s plenty of blame to go around, including a “radically consolidated” publishing industry “driven by sales and marketing teams"; school and district committees for selecting curriculum filled with teachers and others who typically lack the appropriate time, expertise, and motivation to make wise choices; and state textbook adoptions focused on adherence to standards with little or no attention to whether the materials reflect a coherent and well-designed instructional approach.

In other words, if a robust research base existed, would it be put to good use?

Related Tags:

A version of this news article first appeared in the Curriculum Matters blog.

Commenting has been disabled on effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Science Opinion Four Good Science Teaching Strategies & How to Use Them
Three science educators share their "go-to" teaching strategies, including encouraging student talk & implementing project-based learning.
11 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Science Opinion The Three Most Effective Instructional Strategies for Science—According to Teachers
Three science educators share their favorite instructional strategies, including incorporating a sense of play in their classes.
9 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
Science Make Science Education Better, More Equitable, Says National Panel
States must take steps to ensure that all students get a fair shot at learning science, says the National Academies of Science report.
3 min read
Illustration of father and child working on computer.
Science Q&A Many Schools Don't Teach About the Science of Vaccines. Here's Why They Should
Schools play an important role in confronting misinformation and mistrust in vaccines by helping students understand how they work.
7 min read
Ainslee Bolejack, freshman at Shawnee Heights High School in Tecumseh, Kansas, prepares to receive her first COVID-19 vaccine on May 17, 2021, at Topeka High. Unified School District 501 held a clinic at all their high schools welcoming students now 12-years-old and up to receive their vaccination.
Freshman Ainslee Bolejack prepares to receive her first COVID-19 vaccine on May 17, 2021, at Topeka High School in Kansas. Unified School District 501 held a clinic at all its high schools for students 12 and older to receive their vaccinations.
Evert Nelson/The Topeka Capital-Journal via AP