School & District Management Federal File

Statistical Errors?

By Debra Viadero — January 30, 2007 2 min read

Research advisory panel splits over NCES studies of cause and effect

The fallout over a pair of controversial studies released last year by the Department of Education’s chief statistics branch raged on last week when a national research advisory board met in Washington.

The studies—one comparing student achievement in public and private schools, and the other comparing achievement in charter schools with that of regular public schools—caused an uproar last year when they were published by the National Center for Education Statistics. That’s in part because the reports had made statistical adjustments to account for socioeconomic differences between the comparison groups.

Critics, including some top Education Department officials, said the analyses overstepped the NCES’ mission as a purely statistics-gathering agency and jeopardized its credibility.

Among the critics was the congressionally mandated National Board for Education Sciences, the panel that met last week. In September, the board approved a resolution recommending that the NCES refrain from commissioning or publishing studies that “purport” to explore the causal effects of policies. (“‘Physics First’ Is Moving Slowly Into Nation’s High Schools,” Sept. 6, 2006.)

At the board’s Jan. 23-24 meeting, though, the two board members who spearheaded the original resolution said the wording didn’t go far enough. Arguing that the earlier language failed to capture their intent, Eric A. Hanushek and Caroline M. Hoxby, both prominent economists, put forth a new proposal to delete the word “purport” and recommend instead that the NCES avoid studies “that could reasonably be interpreted” to be analyzing policy effects.

Mr. Hanushek, a senior fellow at Stanford University’s Hoover Institution, said the broader terms were needed because the reports had not purported to be cause-and-effect studies, either. They each carried disclaimers cautioning against reading too much into their findings, yet the results were widely misinterpreted to mean that charter schools, or private schools, didn’t work.

The new proposal drew a heated response from Mark S. Schneider, the NCES commissioner and a critic himself of the much-debated reports, which were commissioned before his watch. He said the new proposal was too open to interpretation and ran counter to the federal law prescribing his agency’s mission.

“We’re trying to fix something we all agree was a mistake,” he argued. The board narrowly agreed, voting 6-5 to scratch the new proposal.

Related Tags:

A version of this article appeared in the January 31, 2007 edition of Education Week

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Building Leadership Excellence Through Instructional Coaching
Join this webinar for a discussion on instructional coaching and ways you can link your implement or build on your program.
Content provided by Whetstone Education/SchoolMint
Teaching Webinar Tips for Better Hybrid Learning: Ask the Experts What Works
Register and ask your questions about hybrid learning to our expert panel.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Families & the Community Webinar
Family Engagement for Student Success With Dr. Karen Mapp
Register for this free webinar to learn how to empower and engage families for student success featuring Karen L. Mapp.
Content provided by Panorama Education & PowerMyLearning

EdWeek Top School Jobs

2021-2022 Teacher (Districtwide)
Dallas, TX, US
Dallas Independent School District
[2021-2022] Founding Middle School Academic Dean
New York, NY, US
DREAM Charter School
DevOps Engineer
Portland, OR, US
Northwest Evaluation Association
User Experience Analyst
Portland, OR, US
Northwest Evaluation Association

Read Next

School & District Management The Key to School-Based COVID-19 Testing: Cooperation of Parents and Communities
As schools launch broad testing to track cases of COVID-19, the success of their efforts relies on addressing the concerns of all concerned.
7 min read
Katie Ramirez, left, watches as her mother, Claudia Campos, swabs the mouth of her sister, Hailey, for a COVID-19 test at a testing site in Los Angeles on Dec. 9, 2020.
Katie Ramirez, left, watches as her mother, Claudia Campos, swabs the mouth of her sister, Hailey, for a COVID-19 test at a testing site in Los Angeles.
Jae C. Hong/AP
School & District Management Interactive A Year of COVID-19: What It Looked Like for Schools
This timeline offers a look at how a full year of living and learning during the COVID-19 pandemic unfolded.
Education Week Staff
13 min read
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Elementary 1 teacher Melissa Vozar sits outside of Suder Elementary in Chicago to teach a virtual class on Jan. 11, 2021. The Chicago Teachers Union said that its members voted to defy an order to return to the classroom before they are vaccinated against the coronavirus, setting up a showdown with district officials who have said such a move would amount to an illegal strike.
Anthony Vazquez/Chicago Sun-Times via AP
School & District Management Most Principals, District Leaders Predict Their Schools Will Be Fully In-Person This Fall
EdWeek Research Center surveys track the growing trend to get more students back in school buildings as soon as possible.
5 min read
Assistant Principal Janette Van Gelderen, left, welcomes students at Newhall Elementary in Santa Clarita, Calif on Feb. 25, 2021. California's public schools could get $6.6 billion from the state Legislature if they return to in-person instruction by the end of March, according to a new agreement announced Monday, March 1, 2021, between Gov. Gavin Newsom and the state's legislative leaders.
Assistant Principal Janette Van Gelderen, left, welcomes students at Newhall Elementary in Santa Clarita, Calif., last month. California's public schools could get $6.6 billion from the state if they return to in-person instruction by the end of March.
Marcio Jose Sanchez/AP
School & District Management Opinion Will the Hybrid School Concept Continue After COVID-19?
In an effort to move from triage to transformation, schools should look at how they continue the hybrid model after the COVID-19 vaccine.
7 min read
Hybrid FCG
Shutterstock