If the Obama administration wants to use education strategies that have been proven to work, the U.S. Department of Education’s former research chief says in a provocative new analysis, then perhaps it ought to take another look at curriculum.
The pitch for curriculum-based school improvement strategies came in a new “letter on education” posted yesterday by Grover J. “Russ” Whitehurst, the director of the Brown Center on Education at the Brookings Institution. Whitehurst, you’ll recall, headed the department’s Institute of Education Sciences during the Bush administration.
In true researcher fashion, he makes his case by comparing typical effect sizes for Obama administration education initiatives such as charter schools, merit pay, and early-childhood education, with those for more curriculum-oriented efforts.
For instance, he notes, federally funded evaluations of commonly used mathematics programs have shown that elementary-level programs such as Saxon Math and Math Expressions produce learning gains with an effect size that is .30 standard deviations larger than those for other math programs. The most favorable study of charter schools, in comparison, yields an effect size of .09 standard deviations for that strategy over the student achievement gains found in regular public schools. (Sorry, these effect sizes can’t be translated to a metric that is more understandable.)
Likewise, a rigorous study in India shows that students taught by teachers taking part in merit-pay programs score .15 standard deviations higher than those whose teachers are paid through more traditional pay structures, Whitehurst says. For a more extensive listing, check out the table that accompanies his analysis.
“We conclude that the effect sizes for curriculum are larger, more certain, and less expensive than for the Obama-favored policy levers,” he writes.
So what about common standards? Aren’t those curricular interventions? That may be true, Whitehurst says, but there’s little research to suggest that setting high academic standards may be the way to go. In their own analysis, Whitehurst and his Brown Center colleagues explored whether students in states ranked high by the Thomas B. Fordham Institute and the American Federation of Teachers for having rigorous curriculum standards in math fare well on National Assessment of Educational Progress tests in that subject. They found no such systematic relationship. Whitehurst writes:
The absence of a correlation between ratings of the quality of standards and student achievement and between the difficulty of state standards and student achievement raises the possibility that better and more rigorous content standards do not lead to higher achievement - perhaps standards are such a leaky bucket with respect to classroom instruction that any potential relationship dissipates before it can be manifest."
Whitehurst’s advice is not necessarily to abandon such efforts. In addition to those reforms, he writes, the department should fund many more comparative effectiveness studies like the one that found such large gains for some types of math curricula and publish the results in a way that is more understandable to the public. He also says the Elementary and Secondary Education Act, when it is renewed, ought to require states and districts to review the research evidence for the curricular decisions they make and justify them in light of that evidence.
But what to do, I wonder, if all those studies turn up no effects?