Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Budget & Finance Opinion

Three Tips for Spending COVID-19 Funds in Evidence-Based Ways

By Rick Hess — July 21, 2021 3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.

As schools race to spend the billions of dollars in COVID-19 aid that Washington has provided, I’m troubled by the number of emails I receive, from district staff or board members, that say things like, “Please don’t tell anybody I shared this, but you won’t believe how we’re spending X million dollars . . .” I think part of the problem is that, even where school and system leaders are intent on making good, “evidence-based” decisions, there’s a lot of uncertainty about just what that means in practice.

After all, districts know they’re supposed to make a particular show right now out of spending money in accord with the “evidence.” The American Recovery Plan stipulates that 20 percent of all American Recovery Plan dollars going to local districts should be used to measure the impact of lost instructional time or pay for “evidence-based interventions” that will help compensate for it. Meanwhile, any number of vendors, advocates, and consultants are busy telling school and system leaders that they’ve got just the (evidence-based) thing to help. It can make it tough to sort the wheat from the chaff.

As leaders try to seek out evidence-based practices that will actually be useful, here are a few suggestions drawn from a piece I recently did for Educational Leadership that they should keep in mind.

First off, the evidence in question is frequently less definitive than we might imagine. Even medical researchers, with their deep pockets and fancy lab equipment, change their minds with distressing regularity on things like the risks of cholesterol, the virtues of flossing, or the effects of alcohol. The fact that a study concludes something doesn’t mean that the conclusion is necessarily “true.” This is why research shouldn’t be followed slavishly.

Consider: In 2015, an attempt to replicate 97 studies with statistically significant results found that more than one-third couldn’t be duplicated. This means that, when someone took the original data and reran the study, the results disappeared. And a confidential survey a few years back found that more than 90 percent of psychology researchers confessed to at least one behavior that might compromise their research, such as stopping data collection early because they liked the results or failing to disclose all of a study’s conditions. The bottom line: There are lots of potential pitfalls when we just “follow the research.” All “evidence” isn’t reliable, and determining what to trust requires judgment, acumen, and patience.

Second, the research that gets pursued is often dictated less by educators than by policymakers and funders, who can provide resources, platforms, and professional opportunities that practitioners just can’t match. Researchers are perfectly willing to tap into big state data sets that allow them to tackle broad policy questions—like whether pre-K or charter schooling boosts student achievement—but which don’t say much about how a given district or school should spend its funds.

Third, even when you’ve found solid, relevant research, it’s not always evident what it means to “follow” it. After all, learning that research says something “works” and actually making it work successfully oneself can be two completely different things. In health care, for instance, if a vaccine “worked” when patients got a 200-milligram dosage exactly 24 days apart, that’s how doctors will administer it. If those instructions aren’t followed, the treatment won’t work the way it’s supposed to.

In schooling, implementation is rarely that disciplined—for reasons ranging from contracts to coaching to culture. Yet, it’s vital to understand precisely how to implement a practice in order for it to work and then to do it. When research finds significant effects from a particular practice, coaches or vendors need to be crystal clear on what that means and what it entails—and school and district leaders need to be willing to execute accordingly. Remember: When “evidence-based” practice winds up only loosely modeled on the research, it’s no longer evidence-based.

Given all that, school and system leaders need to ask hard questions about what purportedly “works.” Even when analyses are actually relevant to a district’s needs, they are often imprecise about what programs require, how they’re structured, or how staff are trained. If COVID funds targeted to adopt or expand evidence-based practices are going to deliver, it’s crucial to be clear on what evidence is actually helpful, what it actually says, and how to use it well.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
Meeting the Moment: Accelerating Equitable Recovery and Transformative Change
Educators are deciding how best to re-establish routines such as everyday attendance, rebuild the relationships for resilient school communities, and center teaching and learning to consciously prioritize protecting the health and overall well-being of students
Content provided by Campaign for Grade-Level Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Addressing Learning Loss: What Schools Need to Accelerate Reading Instruction in K-3
When K-3 students return to classrooms this fall, there will be huge gaps in foundational reading skills. Does your school or district need a plan to address learning loss and accelerate student growth? In this
Content provided by PDX Reading
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Budget & Finance Some School Districts Are Feeling COVID-19 Stimulus Envy
Thousands of districts got little to nothing from recent federal stimulus aid, surfacing longstanding tensions over inadequate school funding.
11 min read
An old swing-set on a playground at Heritage Elementary School in Lewis Center, OH on July 7, 2021. New construction and repairs will be paid for by a school levy since the district didn’t qualify for pandemic relief funding.
The Zionsville school district in Indiana is one of roughly 1,000 in the U.S. that received no money from the second and third rounds of federal stimulus aid during the COVID-19 pandemic.
Maddie McGarvey for Education Week
Budget & Finance Quiz Quiz: How Much Do You Know About Education Funding?
Quiz Yourself: How much do you know about what's going on with education funding?
Budget & Finance School Buildings Are Crumbling. Here's Why It's So Hard to Fix Them
As the pandemic comes to an end and districts prepare to fully reopen schools, they face new construction costs and deadline pressures.
10 min read
Image of an excavator in front of a school building.
iStock/Getty
Budget & Finance One Big Reason Schools Are Ditching Remote Learning: The Cost
Despite increasing demand from parents, states are giving districts little financial incentive to provide online learning this fall.
13 min read
Heath Oates, Superintendent of El Dorado Springs R-2 School District, says to drive through the the school district of El Dorado Springs in Missouri "from end to end could take 45 minutes." Much of it doesn't have cell phone service. His school district is all white. Students were given Verizon Hotspots so they could attend virtually.
Heath Oates, superintendent of the sprawling El Dorado Springs R-2 School District in Missouri, says much of the community has spotty Wi-Fi and cell service, so students were given Verizon Hotspots so they could attend school virtually.
Jim Barcus for Education Week