Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Budget & Finance Opinion

Three Tips for Spending COVID-19 Funds in Evidence-Based Ways

By Rick Hess — July 21, 2021 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

As schools race to spend the billions of dollars in COVID-19 aid that Washington has provided, I’m troubled by the number of emails I receive, from district staff or board members, that say things like, “Please don’t tell anybody I shared this, but you won’t believe how we’re spending X million dollars . . .” I think part of the problem is that, even where school and system leaders are intent on making good, “evidence-based” decisions, there’s a lot of uncertainty about just what that means in practice.

After all, districts know they’re supposed to make a particular show right now out of spending money in accord with the “evidence.” The American Recovery Plan stipulates that 20 percent of all American Recovery Plan dollars going to local districts should be used to measure the impact of lost instructional time or pay for “evidence-based interventions” that will help compensate for it. Meanwhile, any number of vendors, advocates, and consultants are busy telling school and system leaders that they’ve got just the (evidence-based) thing to help. It can make it tough to sort the wheat from the chaff.

As leaders try to seek out evidence-based practices that will actually be useful, here are a few suggestions drawn from a piece I recently did for Educational Leadership that they should keep in mind.

First off, the evidence in question is frequently less definitive than we might imagine. Even medical researchers, with their deep pockets and fancy lab equipment, change their minds with distressing regularity on things like the risks of cholesterol, the virtues of flossing, or the effects of alcohol. The fact that a study concludes something doesn’t mean that the conclusion is necessarily “true.” This is why research shouldn’t be followed slavishly.

Consider: In 2015, an attempt to replicate 97 studies with statistically significant results found that more than one-third couldn’t be duplicated. This means that, when someone took the original data and reran the study, the results disappeared. And a confidential survey a few years back found that more than 90 percent of psychology researchers confessed to at least one behavior that might compromise their research, such as stopping data collection early because they liked the results or failing to disclose all of a study’s conditions. The bottom line: There are lots of potential pitfalls when we just “follow the research.” All “evidence” isn’t reliable, and determining what to trust requires judgment, acumen, and patience.

Second, the research that gets pursued is often dictated less by educators than by policymakers and funders, who can provide resources, platforms, and professional opportunities that practitioners just can’t match. Researchers are perfectly willing to tap into big state data sets that allow them to tackle broad policy questions—like whether pre-K or charter schooling boosts student achievement—but which don’t say much about how a given district or school should spend its funds.

Third, even when you’ve found solid, relevant research, it’s not always evident what it means to “follow” it. After all, learning that research says something “works” and actually making it work successfully oneself can be two completely different things. In health care, for instance, if a vaccine “worked” when patients got a 200-milligram dosage exactly 24 days apart, that’s how doctors will administer it. If those instructions aren’t followed, the treatment won’t work the way it’s supposed to.

In schooling, implementation is rarely that disciplined—for reasons ranging from contracts to coaching to culture. Yet, it’s vital to understand precisely how to implement a practice in order for it to work and then to do it. When research finds significant effects from a particular practice, coaches or vendors need to be crystal clear on what that means and what it entails—and school and district leaders need to be willing to execute accordingly. Remember: When “evidence-based” practice winds up only loosely modeled on the research, it’s no longer evidence-based.

Given all that, school and system leaders need to ask hard questions about what purportedly “works.” Even when analyses are actually relevant to a district’s needs, they are often imprecise about what programs require, how they’re structured, or how staff are trained. If COVID funds targeted to adopt or expand evidence-based practices are going to deliver, it’s crucial to be clear on what evidence is actually helpful, what it actually says, and how to use it well.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Attendance Awareness Month: The Research Behind Effective Interventions
More than a year has passed since American schools were abruptly closed to halt the spread of COVID-19. Many children have been out of regular school for most, or even all, of that time. Some
Content provided by AllHere

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Budget & Finance Why Failing to Require Masks Could Cost Districts Millions Later
Some insurance providers are threatening to cancel districts' coverage this school year—particularly if they break statewide mask mandates.
9 min read
Image of a dial that assesses problems, dangers, risks, and liabilities.
iStock/Getty
Budget & Finance Will Teachers Get Vaccinated for $1,000?
More and more districts are offering cash to employees who get vaccinated, hoping that the money will help tamp down COVID-19 spread.
6 min read
Image of a dollar bill folded into an upward arrow.
ImagePixel/iStock/Getty
Budget & Finance How Kids Benefit When Principals Get a Say in Spending Federal COVID-19 Aid
In some districts, principals play a key role in targeting federal pandemic relief money, but in other places they're left out.
8 min read
Nicole Moore, the principal at Indian Mills School, stands near the summer literacy program held in a small lot at Fawn Lake Village in Shamong, New Jersey on July 6, 2021. Moore worked with teachers to develop a summer literacy program for disadvantaged students who live in the district.
Nicole Moore, principal of Indian Mills School, in Shamong, N.J., worked with a teacher and the district superintendent to start a summer program using federal aid for COVID-19 relief.
Eric Sucar for Education Week
Budget & Finance Some School Districts Are Feeling COVID-19 Stimulus Envy
Thousands of districts got little to nothing from recent federal stimulus aid, surfacing longstanding tensions over inadequate school funding.
11 min read
An old swing-set on a playground at Heritage Elementary School in Lewis Center, OH on July 7, 2021. New construction and repairs will be paid for by a school levy since the district didn’t qualify for pandemic relief funding.
The Zionsville school district in Indiana is one of roughly 1,000 in the U.S. that received no money from the second and third rounds of federal stimulus aid during the COVID-19 pandemic.
Maddie McGarvey for Education Week