School Choice & Charters

Reanalysis of NAEP Scores Finds Charter Schools Lagging

August 29, 2006 | Corrected: February 22, 2019 3 min read
  • Save to favorites
  • Print

Corrected: This article presented data on student test scores incorrectly. The study found that 4th grade students in a nationally representative sample of charter schools scored 4.2 points behind students in regular public schools on the National Assessment of Educational Progress in reading, and 4.7 points behind regular public-school students in mathematics, when various student characteristics were considered. Those score differences were measured on the NAEP achievement scale of zero to 500.

A federal reanalysis of 2003 test-score data from the National Assessment of Educational Progress finds that charter schools trailed regular public schools that year in student achievement in both reading and mathematics.

The average reading score for the 150 charter schools examined, taking into account a range of background characteristics of students and schools, was 4.2 percentage points lower than in a pool of more than 6,700 regular public schools, according to the report released last week by the National Center for Education Statistics, an arm of the U.S. Department of Education. In mathematics, the charters scored 4.7 percentage points lower.

“A Closer Look at Charter Schools Using Hierarchical Linear Modeling” is available from the National Center for Education Statistics. Learn more about the NAEP pilot study on charter schools.

The study used data from the 2003 administration of the 4th grade NAEP, which included a special oversampling of charter schools to allow for comparisons between regular and charter public schools. The 150 charters are a nationally representative sample.

Many researchers and charter advocates urge caution in using data from NAEP—the congressionally required assessment of samplings of students in key subjects known as “the nation’s report card”—to examine charter schools. They say, for instance, that the data represent a snapshot at one point in time, with no consideration of students’ prior academic achievement and no way of discerning what effect a charter school has actually had on students.

Using a statistical technique called hierarchical linear modeling, or HLM, the study sought to go further than a similar 2004 NCES comparison of charter and regular public school NAEP scores by considering multiple student and school characteristics simultaneously.

In the new analysis, the differences between charters and regular public schools in the unadjusted data were about 1 percentage point higher in both reading and math than after the student and school characteristics were taken into account.

While the earlier analysis studied student-level scores, the new study focused on the average scores of schools, said Henry I. Braun, a researcher at the Educational Testing Service, of Princeton, N.J., who was the study’s lead author.

Overall, the report’s findings are “numerically very close” to the earlier NCES study, Mr. Braun said. “The statistical significance is more extreme here,” he said, “but in terms of the sort of policy results, there’s not much change after doing this adjustment.”

Results Consistent

During an Aug. 22 conference call with reporters, Mark S. Schneider, the commissioner of the NCES, pointed to a range of caveats about the data, however. “The report was based on 2003 data and may have limited applicability to today’s world of charters,” he said. “About half of the [charter] schools were less than 5 years old.”

Mr. Schneider also believes that the NCES should not undertake studies such as this one, begun under his predecessor, that entail what he sees as subjective methods.

U.S. Secretary of Education Margaret Spellings also emphasized that many charter schools were “relatively new” in an Aug. 22 statement on the study that reflected the Bush administration’s general support of the independent but publicly financed schools. “We need to examine how they improve student performance over time for a better picture of how they compare to traditional public schools,” she said.

Other charter supporters stressed that the study looks at only one year of data.

“The real question about charter school effectiveness is ‘Are kids doing better than they would be doing in other schools?’ ” said Paul T. Hill, who heads the National Charter School Research Project at the University of Washington, in Seattle. “This kind of snapshot study really doesn’t let you answer that question. You can’t sort out what the kid learned in the charter school.”

Nelson Smith, the president of the Washington-based National Alliance for Public Charter Schools, suggested the report is recycling old data.

“This is the fourth time the public has been greeted with essentially the same ‘news’ derived from a source that is singularly ill-suited to the measurement of charter schools’ performance,” he said.

But an official at the American Federation of Teachers, which has become increasingly critical of charter schools and issued its own widely discussed report on the same data two years ago, argues that the new study is very telling.

“They don’t like the results, so they’re dismissing the methodology,” Nancy Van Meter, an expert on charters at the AFT, said of charter proponents. “I think that the preponderance of evidence is coming in saying charter schools are not outperforming public schools on a consistent basis.”

Related Tags:

A version of this article appeared in the August 30, 2006 edition of Education Week as Reanalysis of NAEP Scores Finds Charter Schools Lagging

Events

Classroom Technology K-12 Essentials Forum Making Technology Work Better in Schools
Join experts for a look at the steps schools are taking (or should take) to improve the use of technology in schools.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Budget & Finance Webinar
The ABCs of ESSER: How to Make the Most of Relief Funds Before They Expire
Join a diverse group of K-12 experts to learn how to leverage federal funds before they expire and improve student learning environments.
Content provided by Johnson Controls
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Modernizing Principal Support: The Road to More Connected and Effective Leaders
When principals are better equipped to lead, support, and maintain high levels of teaching and learning, outcomes for students are improved.
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School Choice & Charters Opinion Families May Like Their School But Want More Options. That’s Where Course Choice Comes In
Educational choices have grown inside each school as a result of the pandemic. Families should take advantage of this.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School Choice & Charters Mich. Public School Advocates Launch Effort to Stop DeVos-Backed Proposal
The former secretary of education is backing an initiative that advocates say would create an unconstitutional voucher system.
Samuel J. Robinson, mlive.com
4 min read
Student with backpack.
surasaki/iStock/Getty
School Choice & Charters The Pandemic Pushed More Families to Home School. Many Are Sticking With It
These parents have a common desire to take control of their children's education at a time when control feels elusive for so many people.
Laura Newberry, Los Angeles Times
6 min read
Karen Mozian homeschools her sixth-grade son, Elijah, age 9, at their home in Redondo Beach, California on Jan. 13, 2022. Mozian says her son wasn't getting the kind of help he needed at school. On his study breaks, he enjoys skateboarding and practicing drums.
Karen Mozian homeschools her 6th grade son, Elijah, age 9, at their home in Redondo Beach, California on Jan. 13, 2022. Mozian says her son wasn't getting the kind of help he needed at school. On his study breaks, he enjoys skateboarding and practicing drums.
Carolyn Cole/Los Angeles Times via TNS
School Choice & Charters Bloomberg Launches $750 Million Fund to Grow Charter Schools Amid 'Broken' K-12 System
Former New York City mayor and one-time presidential hopeful Michael R. Bloomberg aims to add 150,000 charter school seats over five years.
5 min read
New York's Mayor Michael Bloomberg, second from left, and Schools Chancellor Dennis Walcott, far left, meeting with senior students at the Bedford Academy High School in New York on Dec. 3, 2013. Bloomberg campaigned on gaining control of the nation's largest public school system. left his mark by championing charter schools, expanding school choice, giving schools letter grades, and replacing scores of struggling institutions with clusters of small schools.
Then-New York City Mayor Michael Bloomberg, second from left, and former Schools Chancellor Dennis Walcott, far left, meet with senior students at Bedford Academy High School in New York in 2013.
Bebeto Matthews/AP