Opinion
School & District Management Opinion

The School Improvement Gains Nobody’s Talking About

Three recommendations for improving education research
By Alan L. Ginsburg, Marshall S. Smith & Phyllis W. Jordan — December 17, 2018 5 min read
BRIC ARCHIVE
  • Save to favorites
  • Print

In Houston, high doses of tutoring helped raise test scores at struggling secondary schools. In San Francisco, extended learning time for students and workshops for parents made a difference. And throughout Massachusetts, greater autonomy for schools and a collegial, collaborative, and professional culture for teachers were among the factors helping to turn around schools.

You won’t find these results in the federal study of the School Improvement Grant program, an Obama-administration initiative that eventually directed as much as $7 billion into low-performing schools across the country.

The 2017 study released by the U.S. Department of Education reported no significant improvement for schools receiving the grants, a finding that U.S. Secretary of Education Betsy DeVos has repeated when she talks about how pointless it is to spend more money on struggling public schools. But that’s not the whole story.

With states set to spend more than $1 billion in federal money in the coming year on school improvement, it’s important that we learn the full lessons of the SIG initiative—and consider revamping how the Education Department evaluates progress.

A shift in the federal research mindset is critical so that we can produce more useful findings to support school improvement under the Every Student Succeeds Act."

In a report from the independent think tank FutureEd, which is based at Georgetown University, we looked beyond the federally funded study at every published state and local study of SIG programs. In 12 of the 17 studies we uncovered, involving about 450 schools receiving grants, these schools made significant gains in student reading or math achievement compared to public schools that didn’t get SIG support. For a typical 3rd or 4th grader, these average gains represent 40 percent of the expected yearly increase in reading achievement and half the expected gain in math.

In California, a study found a dollar spent on SIG grants was more cost effective in improving student achievement than a dollar spent lowering class size. In an Ohio study released this year, SIG schools made significant gains in reading and math compared to schools not in the program. At the local level, SIG schools in San Francisco reduced the achievement gap with other schools in the district by 35 percent and reduced the odds of unexcused absences by 24 percent. And in Ohio and Texas, graduation rates increased by an additional 9 and 5 percentage points, respectively, in high schools that received the grants.

Why did the federal study show no significant gains? We believe that the study’s “statistical power” was so weak that the student performance gains in SIG schools “would have had to be unrealistically large for the study to have been able to detect them,” in the words of Brown University researcher Susanna Loeb. Moreover, the sample of about 190 schools was not nationally representative, so the results can’t be generalized to the nation. And the states they reviewed had widely varying processes for awarding and administering the grants, introducing a lot of static that can drown out positive findings.

The shortcomings of the SIG analysis warrant big changes in the way the federal Education Department and its research arm, the Institute of Education Sciences, conduct program research. A shift in the federal research mindset is critical so that we can produce more useful findings to support school improvement under the Every Student Succeeds Act. Of course, accountability is still important, but we believe studies have their greatest utility when they identify how to improve the neediest schools in each state.

Which brings us back to Houston, San Francisco, and other districts. In contrast to the national study, the state and local studies we reviewed yielded valuable insights into what works in turning around failing schools. And beyond specific strategies, they point to the importance of comprehensive reform work (composed of four or five components) that is supported externally and sustained over multiple years.

These efforts were not reinventing the wheel. The San Francisco work, for instance, was replicating five approaches identified by the University of Chicago Consortium on Chicago School Research, including an emphasis on strengthening school leadership and a student-centered learning climate.

In Houston, Denver, and Lawrence, Mass., SIG initiatives produced positive findings using the strategies that Harvard researcher Roland Fryer found in successful charter schools: increased instructional time, a rigorous approach to building human capital, high-dosage tutoring, frequent use of data to inform instruction, and a culture of high expectations. In fact, the 2014 Houston study found that adding small-group, high-dosage tutoring to reforms increases student outcome gains in secondary schools by 200 percent over their peers without tutoring.

In North Carolina, researchers learned that the required replacement of principals had no effect on teachers’ perceptions of the quality of leadership, perhaps because many of the new principals were inexperienced. Such findings point to the importance of understanding how interventions work within local contexts. It’s hard to learn these lessons from a national study.

Making matters worse, the national study didn’t publish outcomes until after the SIG work was completed; that is the Education Department’s practice, but it meant delay for policymakers and practitioners eager to learn what worked under SIG and why.

All of this calls for a different approach to education research, one based on a federal-state partnership. Toward that end, we make three recommendations:

    1. The U.S. Department of Education should use the school outcome and other data it already collects from every public school in each state to generate annual or biennial state-specific studies of school improvement efforts.

2. The department should help states conduct their own in-depth studies of school improvement using their greater access to school data and knowledge of educational contexts.

3. The department should initiate a National Research Council committee to synthesize the research in the United States and internationally on turning around low-achieving schools.

This fundamentally different approach to studying program interventions would keep a single, nationwide analysis from being the last word on a program playing out differently in schools and districts across the country. And the new approach can inform how valuable education resources could be used more effectively to improve academic success for our most vulnerable students.

Related Tags:

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Classroom Technology Webinar
How to Leverage Virtual Learning: Preparing Students for the Future
Hear from an expert panel how best to leverage virtual learning in your district to achieve your goals.
Content provided by Class
English-Language Learners Webinar AI and English Learners: What Teachers Need to Know
Explore the role of AI in multilingual education and its potential limitations.
Education Webinar The K-12 Leader: Data and Insights Every Marketer Needs to Know
Which topics are capturing the attention of district and school leaders? Discover how to align your content with the topics your target audience cares about most. 

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management After Teachers, America's Schools Spend More on Security Guards Than Any Other Role
New estimates from the Urban Institute indicate school resource officers cost more than $2 billion every year.
4 min read
Illustration of Police silhouettes and a subtle dollar sign to show SRO funding
Wildpixel/iStock
School & District Management Explainer What Does a School Principal Do? An Explainer
Learn about the principal workforce, what makes principals effective, and how schools can retain the best leaders.
Image of staffing.
Andrii Yalanskyi/iStock/Getty
School & District Management Running for a School Board Seat? This Is the Most Powerful Endorsement You Can Get
New research shows that this endorsement in school board races is more influential than any other, with virtually no downside.
5 min read
People in privacy booths vote in the midterm election at an early voting polling site at Frank McCourt High School on the Upper West Side of Manhattan in New York City on Nov. 1, 2022.
People in privacy booths vote in the midterm election at an early voting polling site at Frank McCourt High School on the Upper West Side of Manhattan in New York City on Nov. 1, 2022.
Ted Shaffrey/AP
School & District Management High Pace of Superintendent Turnover Continues, Data Show
About one in five large districts lost a superintendent last year, researchers found.
2 min read
Image of exit doors.
pavel_balanenko/iStock/Getty