Few Studies Track Post-Katrina School Changes
Lost Data, Disparate Efforts Complicated Studies
For education researchers, the waterlogged rubble of New Orleans in 2005 was more than a perfect storm: It was called a perfect opportunity, a “blank slate” on which to test a wide range of education initiatives in a school district already scraping the bottom of the barrel and a community desperate for improvement.
“Two components that made it very compelling,” said Nadya Chinoy Dabby, the director of education programs for the Los Angeles-based Eli and Edythe Broad Foundation, which invested more than $8 million in the years following the storms. “There was the abject, dire need to do something in the wake of the storms … and there was really a chance to remake public education.”
Yet the blank slate has proved much more like an erased chalkboard. Missing data and the ghosts of overlapping improvement efforts make it difficult to get an accurate read on schools’ progress over the long recovery period. Five years later, researchers still struggle to translate the education changes in Crescent City schools into usable models for nationwide reform.
“Every school, the whole system has changed. We have new governance structures, more money than we had before, a new cohort of teachers; we’re paying teachers more, but we have no teachers’ union,” said Shannon Jones, the executive director of the Scott S. Cowen Institute for Public Education Initiatives. “When I think of all the things that have changed, it’s hard to drill down and say what kinds of things work, because there really hasn’t been a control group.”
While research has become a buzzword in school circles, its use in crafting newer, more effective models of schooling remains spotty across New Orleans.
“Some of [the schools] are using research; some of them, not so much,” said Lauren J. Bierbaum, the research director for the Greater New Orleans Afterschool Partnership.
For example, nearly every school in New Orleans has extended the school day, week, or year in some form, but the practices have ranged wildly from school to school, and many are still “reinventing the wheel,” according to Ms. Bierbaum.
“We see a number of schools that are committed to longer days, but they haven’t done the due diligence to understand how you fill that time with something that improves student learning,” Ms. Bierbaum said.
And, while data are being collected on many different fronts, Ms.Bierbaum said, “What we don’t have is a lot of synthesis and people who can talk about it across silos.”
In the immediate aftermath of the hurricanes, philanthropies like the Broad Foundation, the Seattle-based Bill & Melinda Gates Foundation and the Bentonville, Ark.-based Walton Family Foundation funneled money to rebuild and test education reforms.
In 2006, Gloria Ladson-Billings, the then-president of the Washington-based American Education Research Association, said she argued in an impassioned keynote address that the recovery was “a chance to do more than write a check” and that education researchers had a “moral obligation” to study and support the schools’ recovery. Yet the U.S. Department of Education invested no research money into the Gulf, and the expected swarm of independent researchers never materialized.
“New Orleans doesn’t have a major education research school, so there wasn’t a centralized place to get the funding, skills, personnel to look at the unique changes that were taking place here,” said Brian R. Beabout, a University of New Orleans education researcher.
Ms. Ladson-Billings, a professor of curriculum and instruction at the University of Wisconsin-Madison, has been “underwhelmed” by the amount of research in New Orleans so far, but also unsurprised. “People generally were in a research project, and they were reluctant to just drop what they were doing to run down there, and … there was so much disarray.”
Most schools lost student records during the flood or the various administrative changes, meaning there were no baseline data for researchers. Schools changed hands rapidly from year to year and administrators were more concerned with getting classrooms up and running than with answering researcher questions or filling data holes in academic files.
“It would be easy to blame it on Katrina, but I also think the chaos after Katrina opened up some windows for a lot of excuse-making in record-keeping or even not enrolling children in school,” said Kelly McClure, the academic director for the seven charter schools in New Orleans that are operated by the Knowledge Is Power Program, or KIPP.
Early on, Ms. McClure said the KIPP network had an “open-door policy” toward education researchers. “It was very sexy to be doing research on New Orleans, and it was good because it raised awareness that we desperately needed,” Ms. McClure recalled.
Yet Ms. McClure said school administrators grew wary of the agendas behind researchers’ requests and reluctant to put in the time required to handle them. Now, KIPP only opens its doors to research that school leaders think will “get the message out about the reality of the situation.”
Research on the Gulf Coast schools and children that were hit hard during the 2005 hurricane season focuses on topics that run the gamut, from the changing racial demographics to student achievement results of charter schools to post-traumatic stress disorders in preschoolers. Some of the more recent reports include:
The State of Public Education in New Orleans: Five Years After Hurricane Katrina
Scott S. Cowen Institute for Public Education Initiatives at Tulane University in New Orleans (2010)
The report is the latest in a series of annual snapshots of the city’s evolving school network and academic performance. It notes that the proportion of academically unacceptable schools in the city has fallen from 64 percent to 42 percent since 2005, as has the proportion of top-performing “?ve star” schools, from 2 percent to 1 percent.
How Fare the Displaced and Returned Residents of New Orleans? Results of an Innovative Pilot Study
RAND Gulf States Policy Institute in New Orleans and Jackson, Miss. (2010)
The rates of mental illness and dislocation for former New Orleans residents, including families, varies by race and other sociodemographic factors, according to the report.
Charter School Performance
Center for Research on Education Outcomes at Stanford University in Palo Alto, Calif. (2009)
A supplement to the center’s 16-state study of charter schools, the report tracks student data from 2005-06 through 2008-09 to compare the academic growth of demographically matched pairs of students in charters with that of students in other schools in their districts. From the second year and beyond, charter school students performed better in math and reading than did their district peers.
A Portrait of Louisiana: Louisiana Human Development Report 2009
Sarah Burd-Sharps, Kristen Lewis, and Eduardo Borges Martins for the American Human Development Project of the Social Science Research Council in New York City
The report gives detailed indicators of education, health, social services, and economic growth for the entire state before and after Katrina, and the online database associated with the report allows users to compare the indicators with those in the rest of the country.
The State of Public Schools in Post-Katrina New Orleans: The Challenge of Creating Equal Opportunity
Institute on Race and Poverty at the University of Minnesota Law School
Education reforms since Katrina have done little to improve entrenched racial segregation in New Orleans schools, the study found.
Disasters and Their Impact on Child Development
The journal Child Development, August issue
This special section in the journal of the Society for Research in Child Development in Ann Arbor, Mich., includes a series of studies of children’s reactions to stress and trauma after Katrina.
The disaster also drew researchers from the outside with roots in the area. Adrienne Dixson, an associate professor of education at the Ohio State University in Columbus had taught 6th grade at what was then Charles E. Gayarre Elementary School, and she followed her former principal Roslyn Smith to the newly reopened McDonough 42 Elementary Charter School.
“In New Orleans, relationships with people and who you know really matter,” Ms. Dixson said. “You are welcomed when you come there, but people want to know where you’re from, who your mom is, you know? In most cities, you go to the central office and say, ‘I want to do research.’ … Here, you have to negotiate entry school by school.”
A complicated new governance structure, developed after the storms, played a role in that breakdown in communication for researchers. New Orleans includes five types of charter and regular schools, many serving as independent local education agencies, and accountable variously to the Orleans Parish School District, the Recovery School District, and the state board of education.
As district schools convert to charters, they become responsible for the data and reporting requirements normally assigned to districts: budgets and spending, teacher and student demographics, attendance counts, and behavior issues. Schools follow no uniform timeline or format for reporting the information, and some do not track data in the same way from year to year.
To gather reliable data for its annual “State of Public Education in New Orleans” reports, the Cowen Institute enlists staff members and graduate students to spend 20 hours a week calling the 88 schools and 50 operators to get information, Ms. Jones said.
The ease with which schools move from one governance structure to another also undercuts data quality. For example, Douglas High School has changed locations three times as part of the Recovery School District. After continuing to fail, the school closed last year, and this year, KIPP opened its first high school in New Orleans on Douglas’ last school site.
“If you’re a researcher out of, say, Madison, Wis., and you ask for data from the state department of ed, the state official may not know that there’s a new school [at] this site, or there are two schools at that site, or that the same school has changed locations,” Ms. Jones said.
Schools and nonprofit groups also have conducted basic research on their own. KIPP schools now require all new students to attend a three-week summer school where Ms. McClure and other educators conduct assessments to set baselines for the program’s data-heavy model and create or confirm Individualized Education Programs for special education students, where needed. The network is building an online student-data system that will allow educators or researchers to compare student reading and math achievement to attendance rates, IEP goals, and other indicators.
At the same time, Ms. Bierbaum said, foundations like Broad, Gates, and Walton also are transitioning from their initial recovery investment in the city, prompting community discussions on making support more sustainable. The foundations’ approach over the last five years has been largely aimed at managing crises, Ms. Bierbaum explained, so the question is: “How do we ask them to provide more sustainable support?”
The U.S. Education Department’s Investing in Innovation, or i3, grant program, also affords a new opportunity to study how the educational changes in New Orleans can be transferred to other urban districts. With one of the grants announced this month, the nonprofit group New Schools for New Orleans will partner with New Orleans’ Recovery School District and the Achievement School District in Nashville and Memphis, Tenn., to scale up a system of charter school networks for school turnarounds. The plan calls for identifying charter operators with proven success records, enlisting them to take over failing district and charter schools, and then tracking how those approaches affect their students’ academic progress.
According to Neerav Kingsland, the New Schools’ chief strategy officer, the grant provides “an opportunity to take us to a much more rigorous level of research here.”
The city also will hold its first formal conference on post-Katrina education research next month at the University of New Orleans.
“Originally, it was just to report on what is happening in New Orleans,” Mr. Beabout said, but that soon changed when U.S. Secretary of Education Arne Duncan came to the city earlier this year to talk about the rebuilding efforts. “It quickly became apparent that a lot of the logic that undergirded reforms in New Orleans had made it to the White House and was very much a part of the national discussion.”
Mr. Beabout expects the conference to get “a little heated,” as researchers, policymakers, and the community start to hash out what the next five years of reform and research will look like.
Yet, he added, “people are very hopeful about education. Under the old system, it was just scandal after scandal after low test scores after scandals, and people just couldn’t see a way out. Now, people do have hope that the schools can be better.”
At the same time, he said, “New Orleans is becoming more and more a model for things that might happen elsewhere.”
“I think it’s a bit early to do that.” Mr. Beabout added, “because we don’t have the evidence in yet.”
Vol. 30, Issue 01, Pages 13-14Published in Print: August 25, 2010, as Missed Opportunities: Research Scarce on Post-Katrina Reforms