Text-Review Group Shares First Look at Its Process

By Liana Loewus — November 04, 2014 6 min read
Eric Hirsch, far right, the executive director of, talks with educators who are working with his organization to review texts on how well they align with the common standards. The group aims to post its review online this winter.
  • Save to favorites
  • Print

Praise, for many teachers, comes more naturally than criticism.

That dynamic was on full display throughout the weekend training held here by, a nonprofit organization that will begin publishing Consumer Reports-style reviews of curricula and textbooks early next year. The Oct. 25 session offered an early, behind-the-scenes glimpse of the teacher-driven review process, which could eventually have far-reaching consequences for classrooms, district purchasers, and publishers.

Nearly 40 teachers and instructional leaders from across the country gathered for guidance on how to evaluate classroom mathematics materials that claim to be aligned to the Common Core State Standards. One group of reviewers, which has been scrutinizing instructional materials for in small teams since August, participated in “cross-team calibration,” presenting its initial ratings and offering each other feedback. A floor below, a second group—new to the process—learned to use the evaluation tool.

Educators work on evaluating K-8 mathematics materials for their alignment with the Common Core State Standards during a weekend training session last month that was organized by in New York City.

The very-preliminary results shared by the veteran group were variable, to say the least. Some texts they’d reviewed sailed through the first “gateway,” which looks at whether the materials are on grade level and follow the standards’ learning progressions, with high scores. Other curricula received barely a single point on the scoring guide, giving the teachers, despite their best efforts, little to praise. (Education Week has agreed not to disclose preliminary results by publisher.)

“That’s what we struggle with,” said Jennifer Abler, a reviewer and a high school math teacher from Livonia, Mich. “We all want to call out the strengths.”

When the first set of reviews from are made public in February or March, they will look at 21 instructional series for K-8 mathematics from widely used publishers, including Pearson, McGraw-Hill, and Houghton Mifflin Harcourt, as well as a few smaller publishers that were recommended through state review processes. The reviews will be free and easily accessible online.

The group has about $3 million in grant funding, most of which comes from the Bill & Melinda Gates Foundation, the William and Flora Hewlett Foundation, and the Leona M. and Harry B. Helmsley Charitable Trust. (The Gates and Hewlett foundations also help, respectively, to support news coverage of college- and career-ready standards and deeper learning in Education Week.) officially launched in August with 19 curriculum reviewers, about half of whom are classroom teachers. (Not everyone from this group was invited to the New York City training partly because of budget constraints.) It recently added a second cohort with 28 more reviewers, again mostly practicing teachers but also academic coaches and school and district leaders, selected from a larger pool of more than 100 applicants. The reviewers, more than a dozen of whom are National Board-certified, have on average about 14 years of teaching experience. They are paid $2,000 for completing reviews of two instructional series.

Eric Hirsch, the executive director of and its only full-time employee so far, said the group pegged its initial reviews for late winter “to be relevant for purchasing decisions for next year.” The organization also plans to add up to 150 more reviewers in 2015, when it will begin examining high school math and K-12 English/language arts curricula.

Process Up Close

Educators scrutinized a set of mathematics textbooks during one of’s training and evaluation sessions.

Working in teams of four, with one person designated as facilitator, the reviewers evaluate two instructional series, either for grades K-5 or 6-8, over several months. They spend about three or four hours a week looking over the texts on their own, then meet in a weekly video conference to discuss their findings with the rest of the team.

Most of the reviewers also participate in a webinar at some point—keeping their identities hidden—with the publishing companies they’re evaluating to learn more about the print or digital curricula’s features.

Each curriculum is measured against three benchmarks, known as gateways. The first gateway, which a text must pass to move along in the process, determines if it meets the common core’s expectations for focus and coherence—in other words, if it sticks mainly to grade-level content and follows a logical sequence for math learning. The second gateway looks at rigor; the third focuses on usability.

At the recent training, the veteran cohort talked mainly about curricular focus. For example, what if just one or two questions in a textbook were off grade level? Should it get zero credit for that measure?

“It seems very harsh that if one question is off, they get a zero,” Ms. Abler of Michigan said to a small group. Another reviewer chimed in that she wouldn’t fail a student for getting only a single question wrong.

“One thing we’re trying to do is put pressure on the publishers to do better,” replied Melissa Hurt, a curriculum coordinator for the Germantown Municipal district in Shelby County, Tenn. “If the grand purpose is to make a better product, then we need to be harsh.”

Some of the teachers clearly struggled with the process.

“At some point, you feel very bad” when a text continually receives poor scores, said Cassandra Joss, a suburban Detroit kindergarten teacher who works with BetterLesson’s Master Teacher Project, a partnership with the National Education Association. That said, she was steadfast that the scores were justified, for instance, when a team found 6th grade material embedded in 3rd grade lessons.

The highest-scoring materials seemed to warrant the least discussion, though the reviewers did remind each other to steer clear of words like “great” in favor of offering hard evidence for their decisions.

Views From Publishers

What makes this work particularly thorny is that the field hasn’t agreed on what curriculum alignment looks like.

“The more one talks to various education researchers, the more they begin to feel that alignment is to some degree in the eyes of the beholder,” said Jay A. Diskey, the executive director of the Washington-based Association of American Publishers’ P-12 learning group. “We’ve seen cases where certain education researchers look at material and say it’s not aligned, then we see the same materials looked at by states, and they say it is aligned.”

Overall, publishers are eager to see the results of the reviews, said Mr. Diskey, but also feel some anxiety about them.

Comments from some of those publishers varied. While Lynne Munson, the president and executive director of Common Core, a Washington-based nonprofit organization that publishes the Eureka Math curricula, has said she is “extremely excited” about the reviews, a spokesperson from Houghton Mifflin Harcourt declined to comment “on a process we have yet to experience.” Pearson and McGraw-Hill chose not to offer comments.

The reviewers are eager as well—mainly to ensure their efforts will lead to better classroom materials.

“Every educator in my state is constantly asking, ‘What should I be using and how do I know if it’s good?’ ” said Dana Cartier, a math-content specialist at the Illinois Center for School Improvement in Chicago and Naperville, which supports school-improvement efforts across the state. “I think a lot of publishers are going to be fairly unhappy with the results. But that’s why it’s so necessary.”

“The reason I wanted to do this in the first place is the resources we have [in Tennessee] are very shallow, and they don’t reach the depth where we’re supposed to take students,” said Ms. Hurt. “We can help the publishers see that everything they print has an impact. If we are strategic in the way we’re going to present this to companies, they’re more likely to change for the better.”

Related Tags:

A version of this article appeared in the November 05, 2014 edition of Education Week as Group Offers Peek at Text-Review Process


Classroom Technology Webinar How Pandemic Tech Is (and Is Not) Transforming K-12 Schools
The COVID-19 pandemic—and the resulting rise in virtual learning and big investments in digital learning tools— helped educators propel their technology skills to the next level. Teachers have become more adept at using learning management
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Building Teacher Capacity for Social-Emotional Learning
Set goals that support adult well-being and social-emotional learning: register today!

Content provided by Panorama
Jobs October 2021 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Curriculum Holocaust Books Must Be Countered With ‘Opposing’ Views, Texas School Administrator Says
Teachers were told they're required to offer alternative information for debated and controversial topics. The district later apologized.
Brian Niemietz, New York Daily News
2 min read
A book about David Boder's recordings of concentration camp survivors sits next to one of his wire recorders on display at the University of Akron's Center for History of Psychology on Thursday, Feb. 2, 2017 in Akron, Ohio.
A book about David Boder's recordings of concentration camp survivors sits next to one of his wire recorders on display at the University of Akron's Center for History of Psychology on Thursday, Feb. 2, 2017 in Akron, Ohio.
Phil Masturzo/Akron Beacon Journal via AP
Curriculum From Our Research Center Privacy, Porn, and Parents in the Room: Sex Education's Pandemic Challenges
After more than a year of instructional shifts and social isolation, students need sex education that is media-savvy and relationship-wise.
7 min read
Conceptual image of students feeling isolated, but also trying to connect.
Mary Haasdyk for Education Week
Curriculum Calls to Ban Books by Black Authors Are Increasing Amid Critical Race Theory Debates
Books about race and the experiences of Black Americans are being challenged by parents who claim they make white children feel uncomfortable.
8 min read
Fans of Angie Thomas, a Jackson, Miss., resident whose book, "The Hate U Give," has been on a national young adult best-seller list for over 80 weeks, show off their copies at a reception and book signing for the author, in Jackson on Oct. 10, 2018. Thomas' novel has crossed over to a wider audience than simply young adults. The reception honored her writing as well as the coming release of the big screen adaption of the first novel.
The young adult best-seller "The Hate U Give" was one of the top 10 most challenged books of 2020.
Rogelio V. Solis/AP
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Sponsor
Executive Q&A: Building Back Better: The Future of Digital Curriculum
Edmentum’s Chief Strategy Officer Amanda Kocon and Chief Product Officer Todd Mahler discuss the future of digital curriculum.
Content provided by Edmentum Inc.