Federal File

Statistical Errors?

Article Tools
  • PrintPrinter-Friendly
  • EmailEmail Article
  • ReprintReprints

Research advisory panel splits over NCES studies of cause and effect

The fallout over a pair of controversial studies released last year by the Department of Education’s chief statistics branch raged on last week when a national research advisory board met in Washington.

The studies—one comparing student achievement in public and private schools, and the other comparing achievement in charter schools with that of regular public schools—caused an uproar last year when they were published by the National Center for Education Statistics. That’s in part because the reports had made statistical adjustments to account for socioeconomic differences between the comparison groups.

Critics, including some top Education Department officials, said the analyses overstepped the NCES’ mission as a purely statistics-gathering agency and jeopardized its credibility.

Among the critics was the congressionally mandated National Board for Education Sciences, the panel that met last week. In September, the board approved a resolution recommending that the NCES refrain from commissioning or publishing studies that “purport” to explore the causal effects of policies. ("‘Physics First’ Is Moving Slowly Into Nation’s High Schools," Sept. 6, 2006.)

At the board’s Jan. 23-24 meeting, though, the two board members who spearheaded the original resolution said the wording didn’t go far enough. Arguing that the earlier language failed to capture their intent, Eric A. Hanushek and Caroline M. Hoxby, both prominent economists, put forth a new proposal to delete the word “purport” and recommend instead that the NCES avoid studies “that could reasonably be interpreted” to be analyzing policy effects.

Mr. Hanushek, a senior fellow at Stanford University’s Hoover Institution, said the broader terms were needed because the reports had not purported to be cause-and-effect studies, either. They each carried disclaimers cautioning against reading too much into their findings, yet the results were widely misinterpreted to mean that charter schools, or private schools, didn’t work.

The new proposal drew a heated response from Mark S. Schneider, the NCES commissioner and a critic himself of the much-debated reports, which were commissioned before his watch. He said the new proposal was too open to interpretation and ran counter to the federal law prescribing his agency’s mission.

“We’re trying to fix something we all agree was a mistake,” he argued. The board narrowly agreed, voting 6-5 to scratch the new proposal.

Vol. 26, Issue 21, Page 23

Published in Print: January 31, 2007, as Statistical Errors?

Back to Top Back to Top

Most Popular Stories





Sponsor Insights

Free Ebook: How to Implement a Coding Program in Schools

Successful Intervention Builds Student Success

Effective Ways to Support Students with Dyslexia

Stop cobbling together your EdTech

Integrate Science and ELA with Informational Text

Can self-efficacy impact growth for ELLs?

Disruptive Tech Integration for Meaningful Learning

Building Community for Social Good

5 Resources on the Power of Interoperability from Unified Edtech

New campaign for UN World Teachers Day

5 Game-Changers in Today’s Digital Learning Platforms

Hiding in Plain Sight - 7 Common Signs of Dyslexia in the Classroom

The research: Reading Benchmark Assessments

Shifting Mindsets: A Guide for Training Paraeducators to Think Differently About Challenging Behavior

All Students Are Language Learners: The Imagine Learning Language Advantage™

Shifting Mindsets: A Guide for Training Paraeducators to Think Differently About Challenging Behavior

How to Support All Students with Equitable Pathways

2019 K-12 Digital Content Report

3-D Learning & Assessment for K–5 Science

Climate Change, LGBTQ Issues, Politics & Race: Instructional Materials for Teaching Complex Topics

Closing the Science Achievement Gap

Evidence-based Coaching: Key Driver(s) of Scalable Improvement District-Wide

Advancing Literacy with Large Print

Research Sheds New Light on the Reading Brain

Tips for Supporting English Learners Through Personalized Approaches

Response to Intervention Centered on Student Learning

The Nonnegotiable Attributes of Effective Feedback

SEE MORE Insights >