In response to criticism from the math community, EdReports.org, the group that bills itself as the Consumer Reports of common-core instructional materials, is making changes to its textbook review process.
The nonprofit EdReports.org published its first round of reviews in March, looking at K-8 math materials from widely used publishers including Pearson, McGraw-Hill, and Houghton Mifflin Harcourt. Nearly all curricula reviewed were judged as not living up to claims that they are aligned to the common core.
Last month, two professional groups for math educators wrote an open letter criticizing the EdReports.org review process. The National Council of Teachers of Mathematics and the National Council of Supervisors of Mathematics claimed the reviews were incomplete and misrepresent what’s important in the Common Core State Standards.
Yesterday, EdReports.org posted an announcement on its website that it would be making “refinements” to its methodology and reporting process. The group will:
• Tweak its three-tiered “gateway” process. As of now, materials must pass through gateways, or thresholds, in order to continue in the review process. If a text passed gateway 1, which looks at whether it meets the common core’s expectations for focus and coherence, it would be reviewed for gateway 2, which looks at whether the curriculum met the expectations for rigor. Very few materials made it to gateway 2 in the first round of reviews. Starting this summer, EdReports.org will allow more materials, including those that were only “partially met” expectations for gateway 1, to be reviewed for gateway 2. That includes materials that are already on the site, as well as the curricula reviewed going forward.
• Collect more evidence for gateway 1. The first round of materials reviewed were deemed un-aligned if they assessed students on any standards above their grade level. The EdReports.org announcement says it will ask reviewers to collect more evidence on above-grade-level assessment to “ensure our analysis reflects the deep conversations reviewers have to calibrate scores fairly within and across teams.” The group will re-review the 46 textbooks (out of 87 total) that did not receive any points on this measure and revise scores as needed. Four textbooks could potentially meet criteria for gateway 1 and be bumped to gateway 2 if they gain points on this measure, said Eric Hirsch, executive director of EdReports.org.
• Put more information on the website. The group plans to enhance its visuals “to provide a better sense of the strengths and gaps in each series.” EdReports.org will also put its “evidence guides” online so educators can use the tool to conduct their own reviews. It will also give publishers more space to share background information on how the materials were developed and any supplementary services available.
Diane Briars, president of NCTM, responded in an interview that these are “all positive steps.”
“We are pleased they’re going to be revising the existing reviews ... but we’re disappointed some other changes we think are important weren’t made,” she said, including eliminating the gateway process altogether and focusing on the quality of each curricula’s common-core content rather than the percent of time spent on particular standards. “We’re looking forward to continuing to work with them so that ultimately their reviews provide as much information as possible for the field.”