Federal

Guidelines Offered for Meaningful Studies of Achievement in Charters

June 06, 2006 4 min read

A national panel is raising concerns about the quality of much of the research to date on charter school achievement, and has outlined recommendations to help ensure better analyses in the future.

The first in a series of white papers from the group of researchers, issued May 31, notes that while the volume of research on student performance in charters has grown dramatically, the actual quality of that research doesn’t seem to be improving.

“Key Issues in Studying Charter Schools and Achievement: A Review and Suggestions for National Guidelines” is available from the Nationl Charter School Research Project.

“It’s actually fairly discouraging,” Julian Betts, an economics professor at the University of California, San Diego, said at a forum here last week.

The paper was drafted by the Charter School Achievement Consensus Panel, a group convened last year by the National Charter School Research Project at the University of Washington in Seattle.

The analysis concludes that 64 percent of research issued from 2001-03 on charter achievement used research designs that the panel judged to be fair or poor. When the time period was extended to 2001-05, the figures were only marginally better, with 61 percent of studies using fair or poor designs, the paper says.

“The literature has tripled in size between 2003 and 2005,” said Mr. Betts, a panel member who summarized the study May 31 at the Washington-based Brookings Institution. “That’s a huge acceleration. But we don’t see an acceleration of the [quality of] research designs being used.”

The report lays out specific recommendations on how to get the most meaningful results from research into student achievement in the nation’s roughly 3,600 charter schools.

“The bottom line of our white paper, really, is three words: Multiple methods needed,” Mr. Betts said.

No Approach Perfect

The white paper comes amid heated debate about whether charter schools offer a more promising education to students than regular public schools.

The consensus panel’s mission is to evaluate current research on charter school effectiveness and develop standards for future studies. The project also aims to influence the kinds of studies that receive funding and bills itself as a neutral voice in discussions on charter schools.

The panel’s eight members include researchers from different fields with diverse methodological traditions: sociology, economics, psychometrics, and political science. Besides Mr. Betts, they include Paul T. Hill from the University of Washington, who heads up the National Charter School Research Project; Jeffrey R. Henig, a professor of political science and education at Teachers College, Columbia University; and Laura Hamilton, a senior behavioral scientist at the RAND Corp.’s Pittsburgh office.

The paper suggests that the key research question on charter achievement is whether students in the independently run but publicly financed schools are learning more or less than they would have learned in regular public schools.

Of a range of approaches used to study charter achievement, among the strongest is a method that can approximate the conditions of a randomized experiment by comparing students who won a lottery for charter school seats with those who lost out in the lottery, the paper says.

One major limitation of that approach, though, is that it only works for charter schools that are overenrolled and fill seats via lotteries, the panel notes. That raises questions about how representative charter schools in such studies may be.

It said the weakest studies use such methods as comparing average test scores in charter versus noncharter schools using just one year’s test results.

Studies’ Scope Debated

Another weak approach, the study says, is to include controls for few student characteristics. Such a method cannot provide assurance that the charter students are truly similar to the comparison group in regular public schools, the paper says.

“Since no one method is problem-free, the only option is for researchers to use the best methods available to them and make sure the limitations of their results are evident,” the paper says.

Some participants in last week’s forum cautioned about the limits of broad-based comparisons given the wide diversity of charter schools and of state laws governing them, among other factors.

“Charter is a right to be different, a right to be heterogeneous,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation and a former assistant education secretary in President Reagan’s administration. His group, which supports charter schools, recently issued a paper offering a typology of such schools, outlining 55 different categories.

Another participant suggested that the research should also pay attention to how different states weed out good charter schools from bad ones.

Mr. Betts agreed that it’s important to keep in mind the differences among charter schools. He also said there was considerable disagreement among panelists about the value of launching national studies on charter schools.

David Garcia, an assistant professor of education at Arizona State University who came to the event, echoed that point.

“There are no national decisionmakers” for charter policy, he said. “I’m not certain what a national study would tell Arizona that Arizona needs to know.”

A version of this article appeared in the June 07, 2006 edition of Education Week as Guidelines Offered for Meaningful Studies of Achievement in Charters

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Meeting the Moment: Accelerating Equitable Recovery and Transformative Change
Educators are deciding how best to re-establish routines such as everyday attendance, rebuild the relationships for resilient school communities, and center teaching and learning to consciously prioritize protecting the health and overall well-being of students
Content provided by Campaign for Grade-Level Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Addressing Learning Loss: What Schools Need to Accelerate Reading Instruction in K-3
When K-3 students return to classrooms this fall, there will be huge gaps in foundational reading skills. Does your school or district need a plan to address learning loss and accelerate student growth? In this
Content provided by PDX Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal How Political Backlash to Critical Race Theory Reached School Reopening Guidance
A lawmaker wants Miguel Cardona to repudiate the Abolitionist Teaching Network after federal COVID-19 documents referenced the group's work.
6 min read
Rep. Virginia Foxx, R-N.C., is seen at a press conference at the U.S. Capitol on March 9, 2021 in Washington.
Rep. Virginia Foxx, R-N.C., is seen at a press conference at the U.S. Capitol on March 9, 2021 in Washington.<br/>
Graeme Sloan/SIPA USA via AP
Federal Biden Team: Schools Can Go Beyond Trump Rules in Response to Alleged Sexual Misconduct
The Education Department's guidance, released July 20, states that Title IX rules from 2020 lay out "minimum steps" for educators.
3 min read
Symbols of gender.
iStock/Getty
Federal Fact Check: After Furor Over 1619 Project, Feds Adjust History and Civics Grant Plans
A previously obscure history and civics program has weathered a political storm, but what exactly has changed?
4 min read
Education secretary nominee Miguel Cardona speaks during a Senate Health, Education, Labor and Pensions Committee hearing on his nomination on Feb. 3, 2021, in Washington.
Education secretary nominee Miguel Cardona speaks during a Senate Health, Education, Labor and Pensions Committee hearing on his nomination on Feb. 3, 2021, in Washington.
Anna Moneymaker/The New York Times via AP
Federal 'Stop CRT' Bill, Votes in Congress Add to Political Drama Over Critical Race Theory
Sen. Tom Cotton's legislation and votes about critical race theory in the House underscore the issue's potency in Washington.
5 min read
Sen. Tom Cotton, R-Ark., speaks during a hearing to examine United States Special Operations Command and United States Cyber Command in review of the Defense Authorization Request for fiscal year 2022 and the Future Years Defense Program, on Capitol Hill, Thursday, March 25, 2021, in Washington.
Sen. Tom Cotton, R-Ark., speaks during a hearing on Capitol Hill March 25 in Washington.
Andrew Harnik/AP