Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Budget & Finance Opinion

Three Tips for Spending COVID-19 Funds in Evidence-Based Ways

By Rick Hess — July 21, 2021 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

As schools race to spend the billions of dollars in COVID-19 aid that Washington has provided, I’m troubled by the number of emails I receive, from district staff or board members, that say things like, “Please don’t tell anybody I shared this, but you won’t believe how we’re spending X million dollars . . .” I think part of the problem is that, even where school and system leaders are intent on making good, “evidence-based” decisions, there’s a lot of uncertainty about just what that means in practice.

After all, districts know they’re supposed to make a particular show right now out of spending money in accord with the “evidence.” The American Recovery Plan stipulates that 20 percent of all American Recovery Plan dollars going to local districts should be used to measure the impact of lost instructional time or pay for “evidence-based interventions” that will help compensate for it. Meanwhile, any number of vendors, advocates, and consultants are busy telling school and system leaders that they’ve got just the (evidence-based) thing to help. It can make it tough to sort the wheat from the chaff.

As leaders try to seek out evidence-based practices that will actually be useful, here are a few suggestions drawn from a piece I recently did for Educational Leadership that they should keep in mind.

First off, the evidence in question is frequently less definitive than we might imagine. Even medical researchers, with their deep pockets and fancy lab equipment, change their minds with distressing regularity on things like the risks of cholesterol, the virtues of flossing, or the effects of alcohol. The fact that a study concludes something doesn’t mean that the conclusion is necessarily “true.” This is why research shouldn’t be followed slavishly.

Consider: In 2015, an attempt to replicate 97 studies with statistically significant results found that more than one-third couldn’t be duplicated. This means that, when someone took the original data and reran the study, the results disappeared. And a confidential survey a few years back found that more than 90 percent of psychology researchers confessed to at least one behavior that might compromise their research, such as stopping data collection early because they liked the results or failing to disclose all of a study’s conditions. The bottom line: There are lots of potential pitfalls when we just “follow the research.” All “evidence” isn’t reliable, and determining what to trust requires judgment, acumen, and patience.

Second, the research that gets pursued is often dictated less by educators than by policymakers and funders, who can provide resources, platforms, and professional opportunities that practitioners just can’t match. Researchers are perfectly willing to tap into big state data sets that allow them to tackle broad policy questions—like whether pre-K or charter schooling boosts student achievement—but which don’t say much about how a given district or school should spend its funds.

Third, even when you’ve found solid, relevant research, it’s not always evident what it means to “follow” it. After all, learning that research says something “works” and actually making it work successfully oneself can be two completely different things. In health care, for instance, if a vaccine “worked” when patients got a 200-milligram dosage exactly 24 days apart, that’s how doctors will administer it. If those instructions aren’t followed, the treatment won’t work the way it’s supposed to.

In schooling, implementation is rarely that disciplined—for reasons ranging from contracts to coaching to culture. Yet, it’s vital to understand precisely how to implement a practice in order for it to work and then to do it. When research finds significant effects from a particular practice, coaches or vendors need to be crystal clear on what that means and what it entails—and school and district leaders need to be willing to execute accordingly. Remember: When “evidence-based” practice winds up only loosely modeled on the research, it’s no longer evidence-based.

Given all that, school and system leaders need to ask hard questions about what purportedly “works.” Even when analyses are actually relevant to a district’s needs, they are often imprecise about what programs require, how they’re structured, or how staff are trained. If COVID funds targeted to adopt or expand evidence-based practices are going to deliver, it’s crucial to be clear on what evidence is actually helpful, what it actually says, and how to use it well.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

School & District Management K-12 Essentials Forum Get a Strong Start to the New School Year
Get insights and actions from Education Week journalists and expert guests on how to start the new school year on strong footing.
Reading & Literacy Webinar A Roadmap to Multisensory Early Literacy Instruction: Accelerate Growth for All Students 
How can you develop key literacy skills with a diverse range of learners? Explore best practices and tips to meet the needs of all students. 
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Webinar
Supporting 21st Century Skills with a Whole-Child Focus
What skills do students need to succeed in the 21st century? Explore the latest strategies to best prepare students for college, career, and life.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Budget & Finance Inflation Is Putting the Squeeze on Schools. How Are They Coping?
As schools get ready to welcome students back, they're paying more for food, fuel, construction materials, insurance, even paper.
3 min read
Cargo containers chart showing increasing prices of cargo transport
iStock/Getty Images Plus
Budget & Finance Q&A How to Prepare for ESSER Spending Scrutiny: Advice From a Veteran Auditor
Smaller districts, in particular, should be scrupulous in explaining and documenting how they spent their federal COVID-19 aid.
7 min read
Eric Russell, CFO for the city of Columbus, Ohio, pictured on July 15, 2022.
Eric Russell has audited and consulted for more than 100 school districts.
Dustin Chambers for Education Week
Budget & Finance Interest Rates Are Going Up to Fight Inflation. Schools May Feel the Pain Either Way
Rising costs, expected higher wages, and diminished voter support for school levies have district leaders worried.
4 min read
Illustration of a man holding oversized money.
Nuthawut Somsuk/iStock/Getty
Budget & Finance States Grill Districts on How They'll Spend Their ESSER Money for Learning Recovery
The scrutiny is crucial for accountability, but can add delays in using billions in badly needed funds.
6 min read
Image of a clock on a spiral notebook.
iStock/Getty