Happening Today: Live Q&A with Secretary of Education Miguel Cardona. Register to attend.
Opinion
School & District Management Commentary

The School Improvement Gains Nobody’s Talking About

Three recommendations for improving education research
By Alan L. Ginsburg, Marshall S. Smith & Phyllis W. Jordan — December 17, 2018 5 min read
BRIC ARCHIVE

In Houston, high doses of tutoring helped raise test scores at struggling secondary schools. In San Francisco, extended learning time for students and workshops for parents made a difference. And throughout Massachusetts, greater autonomy for schools and a collegial, collaborative, and professional culture for teachers were among the factors helping to turn around schools.

You won’t find these results in the federal study of the School Improvement Grant program, an Obama-administration initiative that eventually directed as much as $7 billion into low-performing schools across the country.

The 2017 study released by the U.S. Department of Education reported no significant improvement for schools receiving the grants, a finding that U.S. Secretary of Education Betsy DeVos has repeated when she talks about how pointless it is to spend more money on struggling public schools. But that’s not the whole story.

With states set to spend more than $1 billion in federal money in the coming year on school improvement, it’s important that we learn the full lessons of the SIG initiative—and consider revamping how the Education Department evaluates progress.

A shift in the federal research mindset is critical so that we can produce more useful findings to support school improvement under the Every Student Succeeds Act."

In a report from the independent think tank FutureEd, which is based at Georgetown University, we looked beyond the federally funded study at every published state and local study of SIG programs. In 12 of the 17 studies we uncovered, involving about 450 schools receiving grants, these schools made significant gains in student reading or math achievement compared to public schools that didn’t get SIG support. For a typical 3rd or 4th grader, these average gains represent 40 percent of the expected yearly increase in reading achievement and half the expected gain in math.

In California, a study found a dollar spent on SIG grants was more cost effective in improving student achievement than a dollar spent lowering class size. In an Ohio study released this year, SIG schools made significant gains in reading and math compared to schools not in the program. At the local level, SIG schools in San Francisco reduced the achievement gap with other schools in the district by 35 percent and reduced the odds of unexcused absences by 24 percent. And in Ohio and Texas, graduation rates increased by an additional 9 and 5 percentage points, respectively, in high schools that received the grants.

Why did the federal study show no significant gains? We believe that the study’s “statistical power” was so weak that the student performance gains in SIG schools “would have had to be unrealistically large for the study to have been able to detect them,” in the words of Brown University researcher Susanna Loeb. Moreover, the sample of about 190 schools was not nationally representative, so the results can’t be generalized to the nation. And the states they reviewed had widely varying processes for awarding and administering the grants, introducing a lot of static that can drown out positive findings.

The shortcomings of the SIG analysis warrant big changes in the way the federal Education Department and its research arm, the Institute of Education Sciences, conduct program research. A shift in the federal research mindset is critical so that we can produce more useful findings to support school improvement under the Every Student Succeeds Act. Of course, accountability is still important, but we believe studies have their greatest utility when they identify how to improve the neediest schools in each state.

Which brings us back to Houston, San Francisco, and other districts. In contrast to the national study, the state and local studies we reviewed yielded valuable insights into what works in turning around failing schools. And beyond specific strategies, they point to the importance of comprehensive reform work (composed of four or five components) that is supported externally and sustained over multiple years.

These efforts were not reinventing the wheel. The San Francisco work, for instance, was replicating five approaches identified by the University of Chicago Consortium on Chicago School Research, including an emphasis on strengthening school leadership and a student-centered learning climate.

In Houston, Denver, and Lawrence, Mass., SIG initiatives produced positive findings using the strategies that Harvard researcher Roland Fryer found in successful charter schools: increased instructional time, a rigorous approach to building human capital, high-dosage tutoring, frequent use of data to inform instruction, and a culture of high expectations. In fact, the 2014 Houston study found that adding small-group, high-dosage tutoring to reforms increases student outcome gains in secondary schools by 200 percent over their peers without tutoring.

In North Carolina, researchers learned that the required replacement of principals had no effect on teachers’ perceptions of the quality of leadership, perhaps because many of the new principals were inexperienced. Such findings point to the importance of understanding how interventions work within local contexts. It’s hard to learn these lessons from a national study.

Making matters worse, the national study didn’t publish outcomes until after the SIG work was completed; that is the Education Department’s practice, but it meant delay for policymakers and practitioners eager to learn what worked under SIG and why.

All of this calls for a different approach to education research, one based on a federal-state partnership. Toward that end, we make three recommendations:

    1. The U.S. Department of Education should use the school outcome and other data it already collects from every public school in each state to generate annual or biennial state-specific studies of school improvement efforts.

2. The department should help states conduct their own in-depth studies of school improvement using their greater access to school data and knowledge of educational contexts.

3. The department should initiate a National Research Council committee to synthesize the research in the United States and internationally on turning around low-achieving schools.

This fundamentally different approach to studying program interventions would keep a single, nationwide analysis from being the last word on a program playing out differently in schools and districts across the country. And the new approach can inform how valuable education resources could be used more effectively to improve academic success for our most vulnerable students.

Related Tags:

Follow the Education Week Commentary section on Facebook and Twitter.

Sign up to get the latest Education Week Commentaries in your email inbox.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Classroom Technology Webinar
Here to Stay – Pandemic Lessons for EdTech in Future Development
What technology is needed in a post pandemic district? Learn how changes in education will impact development of new technologies.
Content provided by AWS

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Principals and Stress: Strategies for Coping in Difficult Times
Running schools in the pandemic has strained leaders in unprecedented ways. Principals share their ideas for how to manage the stress.
6 min read
Illustration of calm woman working at desk
Getty
School & District Management Wanted: Superintendents to Lead Districts Through the End of a Pandemic
Former superintendents say there are signs when it's time to move on. Their replacements are more likely to be greenhorns, experts say.
4 min read
Los Angeles Unified School District Superintendent Austin Beutner speaks at a news conference at the school district headquarters in Los Angeles on March 13, 2020. Beutner will step down as superintendent after his contract ends in June, he announced Wednesday, April 21, 2021.
Austin Beutner, the superintendent of Los Angeles Unified, will step down after his contract ends in June.
Damian Dovarganes/AP
School & District Management Has COVID-19 Led to a Mass Exodus of Superintendents?
This year has been exhausting for superintendents. Some experts say they're seeing an unusually high number of resignations this spring.
5 min read
Chicago Public Schools Superintendent Janice K. Jackson, right, speaks on Feb. 11, 2021, during a news conference at the William H. Brown Elementary School in Chicago. In-person learning for students in pre-k and cluster programs began Thursday, since the district's agreement with the Chicago Teachers Union was reached.
Chicago Public Schools Superintendent Janice K. Jackson, right, announced earlier this week that she would depart the school system. Jackson, who assumed the superintendency in 2018, has worked for more than 20 years in CPS.
Shafkat Anowar
School & District Management Most Schools Offer at Least Some In-Person Classes, According to Feds' Latest Count
A majority of 4th and 8th graders had at least some in-person schooling by March, but inequities persisted.
3 min read
Image shows empty desks in a classroom.
Chris Ryan/OJO Images