Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Budget & Finance Opinion

Three Tips for Spending COVID-19 Funds in Evidence-Based Ways

By Rick Hess — July 21, 2021 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

As schools race to spend the billions of dollars in COVID-19 aid that Washington has provided, I’m troubled by the number of emails I receive, from district staff or board members, that say things like, “Please don’t tell anybody I shared this, but you won’t believe how we’re spending X million dollars . . .” I think part of the problem is that, even where school and system leaders are intent on making good, “evidence-based” decisions, there’s a lot of uncertainty about just what that means in practice.

After all, districts know they’re supposed to make a particular show right now out of spending money in accord with the “evidence.” The American Recovery Plan stipulates that 20 percent of all American Recovery Plan dollars going to local districts should be used to measure the impact of lost instructional time or pay for “evidence-based interventions” that will help compensate for it. Meanwhile, any number of vendors, advocates, and consultants are busy telling school and system leaders that they’ve got just the (evidence-based) thing to help. It can make it tough to sort the wheat from the chaff.

As leaders try to seek out evidence-based practices that will actually be useful, here are a few suggestions drawn from a piece I recently did for Educational Leadership that they should keep in mind.

First off, the evidence in question is frequently less definitive than we might imagine. Even medical researchers, with their deep pockets and fancy lab equipment, change their minds with distressing regularity on things like the risks of cholesterol, the virtues of flossing, or the effects of alcohol. The fact that a study concludes something doesn’t mean that the conclusion is necessarily “true.” This is why research shouldn’t be followed slavishly.

Consider: In 2015, an attempt to replicate 97 studies with statistically significant results found that more than one-third couldn’t be duplicated. This means that, when someone took the original data and reran the study, the results disappeared. And a confidential survey a few years back found that more than 90 percent of psychology researchers confessed to at least one behavior that might compromise their research, such as stopping data collection early because they liked the results or failing to disclose all of a study’s conditions. The bottom line: There are lots of potential pitfalls when we just “follow the research.” All “evidence” isn’t reliable, and determining what to trust requires judgment, acumen, and patience.

Second, the research that gets pursued is often dictated less by educators than by policymakers and funders, who can provide resources, platforms, and professional opportunities that practitioners just can’t match. Researchers are perfectly willing to tap into big state data sets that allow them to tackle broad policy questions—like whether pre-K or charter schooling boosts student achievement—but which don’t say much about how a given district or school should spend its funds.

Third, even when you’ve found solid, relevant research, it’s not always evident what it means to “follow” it. After all, learning that research says something “works” and actually making it work successfully oneself can be two completely different things. In health care, for instance, if a vaccine “worked” when patients got a 200-milligram dosage exactly 24 days apart, that’s how doctors will administer it. If those instructions aren’t followed, the treatment won’t work the way it’s supposed to.

In schooling, implementation is rarely that disciplined—for reasons ranging from contracts to coaching to culture. Yet, it’s vital to understand precisely how to implement a practice in order for it to work and then to do it. When research finds significant effects from a particular practice, coaches or vendors need to be crystal clear on what that means and what it entails—and school and district leaders need to be willing to execute accordingly. Remember: When “evidence-based” practice winds up only loosely modeled on the research, it’s no longer evidence-based.

Given all that, school and system leaders need to ask hard questions about what purportedly “works.” Even when analyses are actually relevant to a district’s needs, they are often imprecise about what programs require, how they’re structured, or how staff are trained. If COVID funds targeted to adopt or expand evidence-based practices are going to deliver, it’s crucial to be clear on what evidence is actually helpful, what it actually says, and how to use it well.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Deepen the Reach and Impact of Your Leadership
This webinar offers new and veteran leaders a unique opportunity to listen and interact with four of the most influential educational thinkers in North America. With their expert insights, you will learn the key elements
Content provided by Solution Tree
Science K-12 Essentials Forum Teaching Science Today: Challenges and Solutions
Join this event which will tackle handling controversy in the classroom, and making science education relevant for all students.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Stronger Together: Integrating Social and Emotional Supports in an Equity-Based MTSS
Decades of research have shown that when schools implement evidence-based social and emotional supports and programming, academic achievement increases. The impact of these supports – particularly for students of color, students from low-income communities, English
Content provided by Illuminate Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Budget & Finance Is Your School District Underfunded?
How access to affordable housing is linked to a K-12 spending gap.
4 min read
Image of architectural model of a neighborhood.
Freder/iStock/Getty
Budget & Finance Schools Had Plans for Federal Relief Aid. The Delta Variant Upended Them
Districts planned to spend federal funds on HVAC upgrades and new technology—but some are now redirecting money to face the Delta variant.
5 min read
Orchard Knob Middle School Assistant Principal Michael Calloway squirts sanitizer onto students hands as they arrive for the first day of school at Orchard Knob Middle School, in Chattanooga, Tenn., on Thursday, Aug. 12, 2021.
Orchard Knob Middle School Assistant Principal Michael Calloway squirts sanitizer on students' hands as they arrive for the first day of classes at the Chattanooga, Tenn., school.
Robin Rudd/Chattanooga Times Free Press via AP
Budget & Finance Why Failing to Require Masks Could Cost Districts Millions Later
Some insurance providers are threatening to cancel districts' coverage this school year—particularly if they break statewide mask mandates.
9 min read
Image of a dial that assesses problems, dangers, risks, and liabilities.
iStock/Getty
Budget & Finance Will Teachers Get Vaccinated for $1,000?
More and more districts are offering cash to employees who get vaccinated, hoping that the money will help tamp down COVID-19 spread.
6 min read
Image of a dollar bill folded into an upward arrow.
ImagePixel/iStock/Getty