Opinion
School & District Management Commentary

The School Improvement Gains Nobody’s Talking About

Three recommendations for improving education research
By Alan Ginsburg, Marshall S. Smith & Phyllis W. Jordan — December 17, 2018 5 min read
BRIC ARCHIVE

In Houston, high doses of tutoring helped raise test scores at struggling secondary schools. In San Francisco, extended learning time for students and workshops for parents made a difference. And throughout Massachusetts, greater autonomy for schools and a collegial, collaborative, and professional culture for teachers were among the factors helping to turn around schools.

You won’t find these results in the federal study of the School Improvement Grant program, an Obama-administration initiative that eventually directed as much as $7 billion into low-performing schools across the country.

The 2017 study released by the U.S. Department of Education reported no significant improvement for schools receiving the grants, a finding that U.S. Secretary of Education Betsy DeVos has repeated when she talks about how pointless it is to spend more money on struggling public schools. But that’s not the whole story.

With states set to spend more than $1 billion in federal money in the coming year on school improvement, it’s important that we learn the full lessons of the SIG initiative—and consider revamping how the Education Department evaluates progress.

A shift in the federal research mindset is critical so that we can produce more useful findings to support school improvement under the Every Student Succeeds Act."

In a report from the independent think tank FutureEd, which is based at Georgetown University, we looked beyond the federally funded study at every published state and local study of SIG programs. In 12 of the 17 studies we uncovered, involving about 450 schools receiving grants, these schools made significant gains in student reading or math achievement compared to public schools that didn’t get SIG support. For a typical 3rd or 4th grader, these average gains represent 40 percent of the expected yearly increase in reading achievement and half the expected gain in math.

In California, a study found a dollar spent on SIG grants was more cost effective in improving student achievement than a dollar spent lowering class size. In an Ohio study released this year, SIG schools made significant gains in reading and math compared to schools not in the program. At the local level, SIG schools in San Francisco reduced the achievement gap with other schools in the district by 35 percent and reduced the odds of unexcused absences by 24 percent. And in Ohio and Texas, graduation rates increased by an additional 9 and 5 percentage points, respectively, in high schools that received the grants.

Why did the federal study show no significant gains? We believe that the study’s “statistical power” was so weak that the student performance gains in SIG schools “would have had to be unrealistically large for the study to have been able to detect them,” in the words of Brown University researcher Susanna Loeb. Moreover, the sample of about 190 schools was not nationally representative, so the results can’t be generalized to the nation. And the states they reviewed had widely varying processes for awarding and administering the grants, introducing a lot of static that can drown out positive findings.

The shortcomings of the SIG analysis warrant big changes in the way the federal Education Department and its research arm, the Institute of Education Sciences, conduct program research. A shift in the federal research mindset is critical so that we can produce more useful findings to support school improvement under the Every Student Succeeds Act. Of course, accountability is still important, but we believe studies have their greatest utility when they identify how to improve the neediest schools in each state.

Which brings us back to Houston, San Francisco, and other districts. In contrast to the national study, the state and local studies we reviewed yielded valuable insights into what works in turning around failing schools. And beyond specific strategies, they point to the importance of comprehensive reform work (composed of four or five components) that is supported externally and sustained over multiple years.

These efforts were not reinventing the wheel. The San Francisco work, for instance, was replicating five approaches identified by the University of Chicago Consortium on Chicago School Research, including an emphasis on strengthening school leadership and a student-centered learning climate.

In Houston, Denver, and Lawrence, Mass., SIG initiatives produced positive findings using the strategies that Harvard researcher Roland Fryer found in successful charter schools: increased instructional time, a rigorous approach to building human capital, high-dosage tutoring, frequent use of data to inform instruction, and a culture of high expectations. In fact, the 2014 Houston study found that adding small-group, high-dosage tutoring to reforms increases student outcome gains in secondary schools by 200 percent over their peers without tutoring.

In North Carolina, researchers learned that the required replacement of principals had no effect on teachers’ perceptions of the quality of leadership, perhaps because many of the new principals were inexperienced. Such findings point to the importance of understanding how interventions work within local contexts. It’s hard to learn these lessons from a national study.

Making matters worse, the national study didn’t publish outcomes until after the SIG work was completed; that is the Education Department’s practice, but it meant delay for policymakers and practitioners eager to learn what worked under SIG and why.

All of this calls for a different approach to education research, one based on a federal-state partnership. Toward that end, we make three recommendations:

    1. The U.S. Department of Education should use the school outcome and other data it already collects from every public school in each state to generate annual or biennial state-specific studies of school improvement efforts.

2. The department should help states conduct their own in-depth studies of school improvement using their greater access to school data and knowledge of educational contexts.

3. The department should initiate a National Research Council committee to synthesize the research in the United States and internationally on turning around low-achieving schools.

This fundamentally different approach to studying program interventions would keep a single, nationwide analysis from being the last word on a program playing out differently in schools and districts across the country. And the new approach can inform how valuable education resources could be used more effectively to improve academic success for our most vulnerable students.

Related Tags:

Follow the Education Week Commentary section on Facebook and Twitter.

Sign up to get the latest Education Week Commentaries in your email inbox.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
How Schools Can Implement Safe In-Person Learning
In order for in-person schooling to resume, it will be necessary to instill a sense of confidence that it is safe to return. BD is hosting a virtual panel discussing the benefits of asymptomatic screening
Content provided by BD
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
How Districts Are Centering Relationships and Systemic SEL for Back to School 21-22
As educators and leaders consider how SEL fits into their reopening and back-to-school plans, it must go beyond an SEL curriculum. SEL is part of who we are as educators and students, as well as
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Webinar
The Fall K-3 Classroom: What the data imply about composition, challenges and opportunities
The data tracking learning loss among the nation’s schoolchildren confirms that things are bad and getting worse. The data also tells another story — one with serious implications for the hoped for learning recovery initiatives
Content provided by Campaign for Grade-Level Reading

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion Have School Reopening Decisions Been Driven by Union Influence?
Rick Hess speaks with Corey DeAngelis about his findings that districts in areas with stronger unions were less likely to reopen in person.
6 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School & District Management Female Principals Are Paid Less Than Men. That’s a Big Concern
A gender-pay-gap in the principalship can affect recruitment and turnover.
5 min read
A conceptual image of gender pay gap.
hyejin kang/iStock/Getty
School & District Management Opinion 42 Percent of Principals Want to Leave Their Position. Will You Let Them?
There are specific reasons leaders cite for reasons they want to leave their positions. What can we do about it? Do we care?
7 min read
42% of Principals are considering leaving their position. Are you one of them?
42% of Principals are considering leaving their position. Are you one of them?
Shutterstock
School & District Management Is the Assistant Principal the Most Overlooked, Undervalued Person at School?
A new research review on assistant principals finds that the role is undefined and that support for these school leaders is inconsistent.
7 min read
 teachers and leaders looking around for direction
Mykyta Dolmatov/iStock/Getty Images Plus