After analyzing dozens of K-12 math textbooks—and determining that most of the major publishers were selling subpar common-core products—the curriculum review group EdReports.org has now moved on to English/language arts materials.
The first round of ELA ratings, released today, were generally more positive than the math ratings have been—however, they were mixed overall. Of the seven instructional series analyzed, three completely met the benchmarks for being considered aligned to the Common Core State Standards for reading and three partially met them. Just one textbook series—Pearson’s Reading Street Common Core for grades 3-6—was deemed fully unaligned.
EdReports.org representatives said the initial sample size was too small to allow for drawing conclusions about the ELA materials market in general. But they also noted that the common-core reading standards may be easier to meet than their math counterparts.
“The standards in math and the standards in ELA are pretty different in their specificity around sequence and structure,” said Eric Hirsch, the executive director of EdReports.org, a nonprofit formed two years ago as a tool to help K-12 educators determine whether curricular materials met the Common Core State Standards. “In ELA, there’s a little more flexibility within them.”
While the math reviews focused mainly on textbooks that had a large portion of the market share, the first round of ELA reviews targeted a smaller but more diverse group of materials. That included texts by major publishers such as Amplify and Houghton Mifflin Harcourt, as well as emerging products, such as a free open education resource called Bookworms, which was created by two university professors.
Bianca Olson, a senior vice president for corporate affairs at Houghton Mifflin Harcourt, whose materials for grades 6-8 only partially met expectations, refuted the review, saying in an email that the EdReports.org “methodology continues to show weakness and inaccuracy in its understanding of standards and instructional methodologies.”
Other publishers were more laudatory of the reviews.
Sharon Walpole, an education professor at the University of Delaware and the co-author of Bookworms, which also partially met for alignment but received a higher raw score than others that partially met, was overall pleased with the EdReports.org review of her independently published 3rd through 5th grade digital curriculum. “To me, it’s a very positive and comprehensive review of our work,” she said. “If you look at the actual report, there’s only two weaknesses [in our curriculum]—one we agree with and one we disagree with. Partially [meets for alignment] seems a little harsh maybe, but I think they used their procedure fairly.”
Walpole and her co-author were also glad to be included in the review alongside the major publishers. “I think it’s pretty amazing that in our offices we could come up on our own with something that competes with them,” she said.
Zachary Goldberg, a spokesman for the College Board, which publishes SpringBoard, said the company was pleased that EdReports.org “recognized the quality and complexity of SpringBoard texts, and also commended the curriculum’s strong writing, research, and grammar components.” Pearson, which had one curriculum that met expectations and one that failed altogether, did not return requests for comment by deadline.
Initial Math Reviews
EdReports.org launched nearly two years ago and put out its first reviews in March 2015. The nonprofit was conceived at the Annenberg Retreat at Sunnylands and spearheaded by Maria M. Klawe, the president of Harvey Mudd College. It is funded by grantors including the Bill & Melinda Gates Foundation—also a major financial backer of the development of the common core—Broadcom Corporation, the Helmsley Charitable Trust, and the William and Flora Hewlett Foundation. (The Gates foundation also supports some coverage of the assessment and implementation of college- and career-ready standards in Education Week and edweek.org.)
In the first round of reviews, which focused on K-8 math products, just one curriculum—Eureka Math, published by Great Minds, a small Washington-based nonprofit organization—was found to be aligned to the common core for all grades.
The other ratings were mostly skewering, with 17 of the 20 math series failing to meet criteria for alignment. Researchers pointed out that the results mirrored those of two previous common-core curriculum studies, which found publishers’ claims of alignment were mostly unfounded.
But the EdReports.org math reviews were also met with substantial backlash. Publishers and the National Council of Teachers of Mathematics said the reviews were incomplete, tainted by shoddy methodology, and ultimately misleading.
“I think they ended up with a lot of false negatives,” Diane J. Briars, then-president of NCTM, said at the time.
EdReports.org heard that feedback, and in October 2015 it tweaked its review process, including by gathering more evidence and putting more information on the website. Several publishers saw improvements in their ratings.
However, that hasn’t completely quelled the pushback. Since then, the popular K-6 math curriculum Everyday Math, which was developed at the University of Chicago and is used in about 200,000 classrooms, received a poor rating. The publisher, McGraw Hill Education, responded that the results conflict with those of other review panels and academic research, which have shown “extensive and undeniable evidence of the strengths and successes of Everyday Mathematics.”
How the Process Works
For both the math and reading reviews, teams of four people—many of whom are current or former practicing teachers—analyzed each textbook series. They met over several months to score the materials using rubrics that EdReports.org devised in consultation with educators and other experts.
The rubrics looked at whether the curricula passed through three “gateways,” or thresholds for performance.
The first gateway for the ELA materials looked at the quality and complexity of the texts students were asked to read. Were the texts both rigorous and engaging? Were they appropriate for the grade level?
“That was always an interesting part of our discussions, talking about the tasks students were doing with the texts,” said Steven Helton, the former director of elementary education with Rutherford County Schools, N.C., and a lead reviewer for EdReports.org. "[We looked to] ensure the students were meaningfully engaged in discussion and not just answering basic questions.”
The second gateway looked at whether the tasks helped students build knowledge and vocabulary. A curriculum needed to meet expectations for both gateway 1 and gateway 2 to be considered aligned. The third gateway rated materials on whether they were user-friendly and provided instructional supports for all kinds of learners.
“We typically didn’t have very much trouble arriving at a consensus,” said Helton.
Are the Reviews Having an Impact?
There’s no precise way to determine how much of an impact EdReports.org is having on local curriculum decisions. But the website, which is free for all users, is approaching 1 million page views, according to Hirsch. And about 150 districts have reached out to the group for help using the reviews.
Jody Guarino, the mathematics coordinator for the Orange County education department in California, and an EdReports.org reviewer for the math materials, said she’s seen districts in her state use the reviews in several ways.
In its most recent adoption process, California approved more than 30 common-core math curricula. “Districts have used EdReports data ... to say let’s narrow to these five,” she said. “It’s almost like a winnowing screener-tool to decrease the list.
They’re also using the reports to find and fill gaps in the curricula they’re already using, and to determine how to support teachers using particular textbooks.
Walpole of Bookworms said while she doesn’t agree with every part of the rubric, the reviews provide enough detail to be really helpful at the local level. “If people read the report, they’d have a very good idea about what our curriculum is and could make a good decision about whether it’s appropriate for them or not,” she said.
EdReports.org is planning to release additional reviews of ELA materials, including some K-2 curricula, this fall.
A version of this news article first appeared in the Curriculum Matters blog.