School & District Management

Adding It All Up

By David J. Hoff — February 19, 2003 10 min read
Are curricula that emphasize conceptual understanding the best way to teach mathematics?

In the 14 years since the National Council of Teachers of Mathematics issued its standards for teaching the subject, a debate has raged over whether schools should follow its recommendations and emphasize conceptual understanding as well as performance of skills.

Curricula based on the national standards have been on the textbook market for six years, and studies now are starting to be published that shed light on the issue of whether the new path was a good one to take.

A panel convened last year by the National Research Council is evaluating the research to determine whether it is rigorous enough to draw conclusions about the effectiveness of the new math curricula. The committee is scheduled to release its report this year.

And the Bush administration is going to be evaluating curriculum in its five-year effort to raise the quality of the nation’s math and science education.

But many math education researchers say that studies published in professional journals and a recent book suggest that the new curricula are on the right track.

The research fails, however, to answer definitively the big question in the debate: Are curricula that emphasize conceptual understanding the best way to teach mathematics? Or should schools continue the traditional approach of teaching the rudimentary skills of the discipline before expecting students to apply them in real-life situations?

The research conducted to date isn’t specific enough to identify which kind of curriculum works best in schools.

“A superintendent of a large school district contacted me not long ago to ask if there was research that would help him select an elementary school math

curriculum that would be effective for the types of children served by his district,” Grover J. “Russ” Whitehurst, the director of the federal Institute of Education Sciences, told a U.S. Department of Education “summit” on math education this month.

“I had to tell him that there was no rigorous research on the efficacy of widely available elementary mathematics curricula, and that about all I could offer him was my opinion,” he said.

Supporters of the new curricula, though, say that enough research is showing positive results for schools to go ahead with the new programs while researchers define the ideal conditions for using them.

“I haven’t seen any evidence that they are failures and we should pull them out of schools,” says James Hiebert, a professor of education at the University of Delaware, located in Newark. “They are promising enough that we should pursue the implementation of them so we can collect long-term data.”

But critics maintain that the research hasn’t delivered nearly enough evidence to warrant schools’ switching to approaches that they say gloss over basic mathematical skills.

“I haven’t seen a study that really convinced me,” says Michael McKeowen, a co-founder of Mathematically Correct, the national parent group that is influential in organizing opposition to the new curricula, and a professor of medical science at Brown University.

Critics maintain that the research hasn't delivered nearly enough evidence to warrant schools' switching to approaches that they say gloss over basic mathematical skills.

The studies often compare the results from new curricula with those from control groups whose teachers didn’t receive the same professional development, McKeowen says. Students who have the chance to study under teachers with such preparation, he says, are likely to perform better than those who learn from less qualified teachers.

While advocates on both sides of the debate differ on the best way to improve math education, there is a consensus that U.S. student performance in the subject needs to improve.

Even though 4th and 8th graders’ scores on the National Assessment of Education Progress rose steadily during the 1990s, U.S. students have scored poorly on several international studies.

In 1996, for example, the Third International Mathematics and Science Study found that U.S. 4th graders performed above the national average, that 8th graders were in the middle of the pack, and that high school seniors fell below average.

A repeat of the study given to 8th graders in 1999 found the 8th graders again to be around the international mean. (“U.S. Students’ Scores Drop by 8th Grade,” Dec. 13, 2000.)

That disappointing record adds urgency to the challenge for researchers: What does work best in math classrooms?

Control Groups Difficult

One study examining the Interactive Mathematics Program, one of the high school curricula influenced by the NCTM standards, paints a picture of how math classes are changing in some American high schools—and gives an example of how difficult it is to draw conclusions from one project.

When freshmen started at a suburban Philadelphia high school in 1997, the study notes, their math classes were different from the classes that had preceded them.

Instead of just teaching algebra—the common 9th grade math subject—the new Interactive Mathematics Program, known as IMP, included pieces of geometry and statistics. Students would have waited until 10th grade or later to learn such subjects under the previous curriculum.

The teaching methods used in the new freshman course also differed markedly from the problem-solving with x’s and y’s representing mythical variables that had dominated the high school’s Algebra 1 course before 1997. Now, teachers offered examples of problems from real-life situations. Students might be asked to find the length of shadows or graph the population growth of the West during the 19th-century migrations.

That same class of 9th graders went through two more years with the same curriculum, along the way supplementing their knowledge of algebra, geometry, and statistics with trigonometry, basic calculus, and other areas of the discipline that traditionally were taught in separate high school courses.

By the time the group reached the end of 11th grade, the researcher who tracked them found that they outperformed a similar group of students, two years ahead of them, who had studied under a traditional curriculum. What’s more, the students in the IMP group, as a whole, had taken more math classes and enrolled in more Advanced Placement courses than those ahead of them.

“It’s a piece of evidence pointing toward a positive effect of the curriculum,” says Steven L. Kramer, who conducted the study as his doctoral dissertation at the University of Maryland and is preparing the results for submission to peer-reviewed journals.

What the research literature lacks, say critics of the new curricula, is studies comparing student achievement under two specific curricula.

Kramer adds that it’s hard to identify whether the curriculum itself was instrumental in raising student achievement or whether it was one of several ingredients needed.

The students who learned under the Interactive Mathematics Program also studied in a block schedule, in which they attended math classes for 90-minute periods over a semester. Their predecessors had attended 45-minute classes that lasted over a whole school year. Kramer’s study was unable to conclude whether IMP would have succeeded under the previous schedule.

Also, the school, which Kramer has not named, decided to switch to IMP with the full support of the math faculty. The teachers all were given training in how to teach the new curriculum.

Would the program have succeeded with a reluctant and unprepared staff of teachers? Common sense and other research suggest not.

That’s why Kramer and others suggest that his research needs to be viewed along with other studies with similar results to decide the effectiveness of the curriculum and the best circumstances for using it.

While studies such as Kramer’s are common, critics of the NCTM standards and the curricula based on them say the research doesn’t offer any evidence that the math innovations are working.

In such studies and others like it, the critics say, the teachers and students are aware that they are research subjects—an awareness that gives them an incentive to perform well.

“These are weak designs because the very act of volunteering to use a new curriculum typically carries with it extra motivation to succeed, and thus biases the results towards the new curriculum,” Whitehurst, the Education Department’s research chief, argued at the math summit.

What the research literature lacks, according to Whitehurst and critics of the NCTM standards and the curricula based on them, is studies comparing student achievement under two specific curricula.

“They should exist, but they don’t exist,” says Bastiaan J. Braams, a research associate professor of mathematics at New York University and a critic of the NCTM.

The research on the new curricula also is hampered, Braams says, by the fact that it has been conducted by people involved in designing the programs or who are supportive of them.

“In general, I have a very low opinion of the research,” he says. “I see it as advocacy research.”

‘Promising and Substantial’

But supporters of the NCTM approach contend that both the quality and quantity of research they are producing provide evidence that the changes are having a positive impact.

In Standards-Based School Mathematics Curricula: What Are They? What Do Students Learn?, published this year, researchers published a collection of 13 studies that they say supports the new curricula.

“The evidence at hand is both promising and substantial,” Jeremy Kilpatrick, a professor of mathematics education at the University of Georgia, in Athens, writes in the book’s concluding essay. “It will not quiet the critics ... but it should encourage those who welcome improvements in school mathematics and understand the difficulty of evaluating something as complex as a curriculum.”

Supporters of the new approach contend that both the quality and quantity of research they are producing provide evidence that the changes are having a positive impact.

In three studies of high schools using IMP, for example, Norman L. Webb found that students who studied under the curriculum performed as well on standardized tests and the SAT as students in a control group. He also found that students using IMP outperformed others when presented with tests that required them to use problem-solving rather than basic skills. They also took more math classes than did students studying a traditional approach, he says.

“We were able to show that the curriculum did what it said it was going to do,” says Webb, a senior research scientist at the Wisconsin Center for Education Research at the University of Wisconsin-Madison.

But it’s still difficult for him to declare IMP a victor in a competition between mathematics curricula, because true control groups are hard to maintain. In Webb’s study, he found that students would transfer from IMP to classes using the traditional curriculum, depending on their interests. Teachers who taught the traditional way started to use techniques, such as small-group projects, that are a common approach in IMP. They also altered what they taught to match the tests they knew researchers would be evaluating.

“It’s very complicated to come up with a good evaluation design,” Webb says.

McKeowen, the co-founder of Mathematically Correct, says that such complications make it tricky to reach definitive judgments about the efficacy of the new curricula.

He agrees that any study that tries to create a separate control group is bound to have the types of problems that Webb cites. The situation is unlike that in a drug study, he point outs, where participants do what they’re told because they don’t know whether they’re taking a placebo or the drug that’s being tested.

New Federal Interest

What is clear is that the debate over research into the new curricula is about to heat up.

The Education Department on Feb. 6 launched an initiative to highlight the need for improved mathematics instruction. Raising the quality of research in the field is a primary objective in the five-year effort. At the mathematics summit in Washington, department officials proposed to spend $120 million in the next fiscal year to conduct research to determine best practices for teachers, as well as to compare curriculum against one another. (“Ed. Dept. Proposes $120 Million Math Agenda,” Feb. 12, 2003.)

What will it take, then, to determine the effectiveness of the new curricula?

‘Expecting a firm or solid proof [in math education] is unrealistic. It's unrealistic in a lot of other areas that people accept. We should expect the same thing in education.’

Kramer, the researcher who studied students at the suburban Philadelphia high school, compares his study to a piece of evidence introduced in court. Alone, the evidence isn’t enough to prove guilt or innocence. But viewed along with other evidence, it can contribute to delivering a verdict, he says.

His study of the Interactive Mathematics Program gives hints, he says, about under which conditions students learn most effectively that are, in turn, supported by similar research.

Hiebert, the University of Delaware professor, suggests that educators and mathematicians need to live with the ambiguity of the research, just as people do in other fields. Recommendations on diet and exercise, he says, are always being updated based on new studies. And when new nutrition standards are introduced—such as suggestions to eat five fruits and vegetables every day—they are guidelines rather than hard-and-fast rules guaranteed to produce better health.

“Expecting a firm or solid proof [in math education] is unrealistic,” Hiebert says. “It’s unrealistic in a lot of other areas that people accept. We should expect the same thing in education.”

The Research section is underwritten by a grant from the Spencer Foundation.

Related Tags:


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Making Digital Literacy a Priority: An Administrator’s Perspective
Join us as we delve into the efforts of our panelists and their initiatives to make digital skills a “must have” for their district. We’ll discuss with district leadership how they have kept digital literacy
Content provided by
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
How Schools Can Implement Safe In-Person Learning
In order for in-person schooling to resume, it will be necessary to instill a sense of confidence that it is safe to return. BD is hosting a virtual panel discussing the benefits of asymptomatic screening
Content provided by BD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
How Districts Are Centering Relationships and Systemic SEL for Back to School 21-22
As educators and leaders consider how SEL fits into their reopening and back-to-school plans, it must go beyond an SEL curriculum. SEL is part of who we are as educators and students, as well as
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion The Year of Scourges: How I Survived Illness and Racism to Find My 'Tribe'
A Black school leader reflects on the hardest year of her professional life.
Reba Y. Hodge
4 min read
new growth on a bare tree
Vanessa Solis/Education Week & Getty Images
School & District Management From Our Research Center How the Pandemic Is Shaping K-12 Education (in Charts)
Surveys by the EdWeek Research Center show how schools have changed during the pandemic and what adjustments are likely to stick.
Eric DiVito gives breathing instructions as he teaches a remote music class at the Osborn School on Oct. 6, 2020, in Rye, N.Y.
Eric DiVito gives breathing instructions as he teaches a remote music class at the Osborn School in Rye, N.Y., last fall.
Mary Altaffer/AP
School & District Management Opinion Ed. Leaders: Discuss Race, Call Out White Supremacy
Downplaying the realities of racism leads to misunderstanding school problems and developing inadequate solutions.
John B. Diamond & Jennifer Cheatham
5 min read
Hand writing the word racism on blackboard. Stop hate. Against prejudice and violence. Lecture about discrimination in school.
Tero Vesalainen/iStock/Getty
School & District Management 'You Can’t Follow CDC Guidelines': What Schools Really Look Like During COVID-19
All year, some teachers have said that enforcing precautions to slow the spread of the virus in classrooms can be nearly impossible.
13 min read
Guntown Middle School eighth graders walk the halls to their next class as others wait in their assigned spots against the wall before moving into their next class during the first day back to school for the Lee County District in Guntown, Miss on Aug. 6, 2020.
Eight graders walk the halls on the first day back to school in Guntown, Miss., on Aug. 6, 2020. Teachers in several states told Education Week that since the beginning of the school year, enforcing precautions such as social distancing to slow the spread of the coronavirus has been nearly impossible.<br/>
Adam Robison/The Northeast Mississippi Daily Journal via AP