Federal Statistics Commissioner Questions NCES Involvement in Private vs. Public School Study

By Sean Cavanagh — August 10, 2006 6 min read
  • Save to favorites
  • Print

Includes updates and/or revisions

The commissioner of the statistical arm of the U.S. Department of Education says his office should not have initiated a recent, heavily publicized study comparing the academic performance of public and private school students because the report relied on a subjective analysis that could lead outsiders to question the research center’s impartiality.

Mark S. Schneider, the commissioner of the National Center for Education Statistics, said last week that he was not faulting the study’s accuracy or methodology. But he said it was not proper for the research office to have directed a study that went so far in making judgments about how to interpret raw school data.

He also said he had the same concerns about the NCES directing a similar study of charter school performance, expected to be released later this month.

“This is not what we should be doing,” Mr. Schneider said of the study of public and private schools, in an interview. “It’s one thing for [an academic researcher] to put out a study. That’s a totally different story than if NCES, as a statistical agency, is weighing in on that front.”

The commissioner voiced his reservations about NCES’ involvement in the public and private school study at an Aug. 4 meeting of the National Assessment Governing Board in McLean, Va., and elaborated on those comments in an interview with Education Week afterward. The governing board is the independent body that sets policy for the National Assessment of Educational Progress, or NAEP, which the NCES administers.

The report on public and private schools, which received widespread attention when it was released last month, showed public school students outperforming their private school peers in reading and mathematics, when researchers adjusted for race, socioeconomic status, and other characteristics. The findings were vigorously challenged by supporters of private schools and vouchers, who argued that the study’s results would have been different if other student characteristics had been chosen.

Mr. Schneider had no say in NCES’ decision to conduct the private school and charter school studies. Those projects were initiated by Robert Lerner, Mr. Schneider’s predecessor. Mr. Lerner could not be reached for comment.

The NCES is one of three researcher centers housed within the Institute of Education Sciences, an independent federal research agency established by Congress in 2002. Although the institute is a part of the U.S. Department of Education, it is supposed to conduct its work free of political influence. The IES director can prepare and publish any research without the approval of the education secretary. While the directors of two of the institute’s research centers are appointed by the IES director, the NCES commissioner is appointed by the president. Both Mr. Lerner and Mr. Schneider were appointed by President Bush; while Mr. Lerner was never confirmed by the U.S. Senate, Mr. Schneider was confirmed last October. His term expires in June 2009.

A Departure

Henry Braun, a senior educational researcher for the Educational Testing Service, a nonprofit research and testing organization in Princeton, N.J., led the public and private school study for the federal statistical agency. It relies on a popular technique known as hierarchical linear modeling, or HLM, which allows researchers to evaluate not only overall data from schools, but also that of smaller, individual groups of students with different characteristics. Using that technique, however, requires researchers to choose which variables to consider—such as race, gender, and various measures of socioeconomic status—as the ETS did on the study of public and private schools. Although it is common for outside researchers to crunch federal data using the HLM method, the NCES has typically not engaged in such work, Mr. Braun and others said.

“I think it was clear from the beginning that this was something of a departure for NCES,” said Mr. Braun of his study. He said he did not have an opinion on whether the statistical agency should direct research using those methods.

Mr. Schneider emphasized that he was not questioning the merits of that NCES report. “It is a very high-quality study.” But the commissioner said in the future he would not support having the NCES produce a report in which researchers made similarly subjective judgments about which student characteristics to consider. A few weeks after the publication of Mr. Braun’s study, Paul E. Peterson, a professor of government at Harvard University, analyzed the data using different variables and found that private school students’ performance topped that of public school students. (“Data Reanalysis Finds Test-Score Edge for Private Schools,” Aug. 9, 2006.)

“Our job is to collect the data and get it out the door,” Mr. Schneider said in the interview. The problem, he said, is that in studies like the private school report, “you’re always having to make choices about variables,” he added. “It’s not something where the models spring from heaven.” The statistical center’s message to the research community, Mr. Schneider said, should be, “we’ll give you the data you want. What you do with it is your business.”

The upcoming NCES-sponsored study of charter schools relies on similar methodology, Mr. Schneider noted. Recent studies of charter schools, using federal data, have ignited fierce debates between defenders of those programs and critics, including teachers’ unions.

“I know there will be a firestorm in three weeks,” Mr. Schneider told the governing board, in reference to the charter school report.

‘Provide the Raw Material’

The director of the Institute of Education Sciences, the independent federal research agency that oversees the statistical center echoed Mr. Schneider’s opinion. In an e-mail, IES Director Grover J. “Russ” Whitehurst, said studies using the HLM and similar analytic methods are “not a priority” for the NCES, because they take time away from other projects, and private-sector researchers have the ability to conduct the same work. Mr. Whitehurst also noted that such interpretive reports almost always produce debates about the methodology used.

“There are typically several legitimate ways to carry out complex data analyses,” Mr. Whitehurst wrote in an e-mail. “Experts will differ on which approach is best and how the results should be interpreted. … Advocacy groups that don’t like the results will frequently assert that the analytic approach was biased. It is best for NCES, as a federal statistical agency, to stay out of that fray.”

Gerald E. Sroufe, the director of government relations for the American Educational Research Association, a professional association in Washington, said he agreed that the NCES should avoid the sort of analysis put forward by the study of public and private schools. Such projects leave the research center open to charges that it is harboring an agenda, said Mr. Sroufe, whose organization represents 25,000 educators and researchers.

“It should provide the raw material,” Mr. Sroufe said of the NCES. Having the statistical center produce studies using HLM and similar methods “is probably not a good idea, particularly involving work that is so politically sensitive,” he said.

The study on public and private schools came out the week before Secretary of Education Margaret Spellings and Republican lawmakers unveiled a new, $100 million proposal to make vouchers for private school tuition available under the No Child Left Behind Act. Critics have cited the NCES report showing public school students outperforming their private school peers, in questioning the wisdom of that voucher proposal.

Despite those debates, Mr. Schneider said his concerns about the NCES’ involvement with the private school study were his own, and that Bush administration officials did not try to influence his views on the subject.

“The NCES,” he said, “is quite insulated from that kind of political pressure.”


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment State Test Results Are In. Are They Useless?
While states, districts, and schools pore over data from spring 2021 tests, experts urge caution over how to interpret and use the results.
9 min read
FILE - In this Jan. 17, 2016 file photo, a sign is seen at the entrance to a hall for a college test preparation class in Bethesda, Md. The $380 million test coaching industry is facing competition from free or low-cost alternatives in what their founders hope will make the process of applying to college more equitable. Such innovations are also raising questions about the relevance and the fairness of relying on standardized tests in admissions process.
A sign is posted at the entrance to a hall for a test-preparation class. Assessment experts say educators should use data from spring 2021 tests with caution.
Alex Brandon/AP
Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Assessment Whitepaper
Proven Techniques for Assessing Students with Technology
Dr. Doug Fisher’s proven assessment techniques help your students become active learners and increase their chances for higher learning g...
Content provided by Achieve3000
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP