Satisfying ESSA's Evidence-Based Requirement Proves Tricky
McDermitt Combined Schools in the rural, northeastern corner of Nevada has been stuck in the academic doldrums for years. The school, which serves a high proportion of low-income Native American students, was flagged under the now-defunct No Child Left Behind Act and got nearly $140,000 in extra federal resources under the School Improvement Grant program.
But the school continued to founder, said David Jensen, the superintendent of the 3,400-student Humboldt school district.
Now, the district is taking advantage of flexibility in the Every Student Succeeds Act that allows superintendents like Jensen, with an assist from their states, to come up with their own school improvement strategies, backed by evidence, to help schools like McDermitt.
Nevada, which is one of the first states to embrace this new approach, helped hook Jensen and his team up with nationally recognized school improvement experts who are providing McDermitt with coaching, helping it to analyze student data, and more. It's a significant—and, Jensen says, welcome—departure from the list of federally prescribed interventions of the past.
But big questions loom as persistently struggling schools around the country prepare to embark on a new school kind of school improvement journey. Will districts and schools use the new leeway in ESSA to find new solutions? Or will they keep doing the same things that they always have, with the same results? How seriously will states and districts take ESSA's evidence-based requirement, if there isn't a ton of federal oversight?
Experts see some promise in the new approach—and plenty to worry about.
"I'm concerned that people in the field will forget about the need to choose really good, evidence-based strategies and implement them really well," said Caitlin Scott, the manager for research and evaluation at Education Northwest, a nonprofit that works on school improvement. "I think that is a danger. ... It's easy to latch onto the next new, shiny, impressive thing without really doing the background-information gathering to see what the evidence is that something is going to be successful."
Lack of Capacity
The biggest potential challenge: capacity.
ESSA, the latest version of the main federal K-12 law, got rid of the NCLB law's controversial School Improvement Grant program, last funded at $450 million in fiscal 2016. But ESSA continued an Obama administration requirement that states identify schools performing in the bottom 5 percent. ESSA directs states to set aside at least 7 percent of their Title I money, which is geared toward disadvantaged students, for school improvement, up from an allowable 4 percent under the old law.
Low-performing districts that get a piece of the federal cash must choose an intervention that's backed by what the law defines as either "strong," "moderate," or "promising" evidence. And districts that don't get federal funding to fix their lowest-performing schools must pick a plan that has a rationale behind it and then study it carefully to see if it's working.
But district leaders may not have the time—or know-how—to take a hard look at the wide universe of potential solutions for low-performing schools and figure out which will really help the students they serve, experts say. And studying the impact of interventions could also prove tricky.
The Every Student Succeeds Act puts districts, states, and schools in the driver’s seat on school improvement. Instead of following federally-required strategies, local leaders can come up with their own interventions for struggling schools. But crucially, these must be backed by evidence.
- States must reserve at least 7 percent of their Title I funds for disadvantaged students for school improvement, and allocate that money to districts.
- These dollars can either be given out competitively, or by formula. At least 14 states plan to give them out competitively.
- If school districts use federal funding for school improvement, they must come up with a plan that is backed by “strong,” “moderate,” or “promising” evidence.
- If schools don’t use federal funding, they must pick a plan that has a rationale behind it, and study it to see if it’s working.
The law requires these evidence-based interventions to be used in two broad categories of schools.
Schools in Comprehensive Improvement
This category applies to the schools considered among the bottom 5 percent of performers in the state, as determined by a mix of factors, including test scores, those where more than a third of students don’t graduate, and those where subgroups of students are chronically underperforming. Initially, the district comes up with a plan to fix these schools, monitored by the state.
Schools in Targeted Improvement
Schools where any particular subgroup of students—English-language learners, students in special education, racial or ethnic minorities, or disadvantaged children—are struggling. The school comes up with a plan to fix these schools, monitored by the district.
"The ask here is actually a really big ask for states and districts," said Carlas McCauley, who ran the School Improvement Grant program as a career official at the U.S. Department of Education during the Obama administration and is now the director of WestEd's Center on School Turnaround in Sacramento, Calif., a research and consulting organization. "It's going to take time to completely move" in this new direction, McCauley said.
Outside the 50 largest urban districts, the person who handles school improvement may be the same staffer who deals with other federal programs centered on factors like teacher quality and school safety. That means districts may be tempted to take outside consultants and providers at their word when they say a strategy meets ESSA's evidence requirements, said Terra Wallin, who also worked as a career staffer at the federal Education Department on school turnaround issues and is now a consultant with Education First, a policy organization that is working with states on ESSA implementation.
"My guess is, you'll see a lot of people doing the things they were already doing," Wallin said. "You'll see a lot of providers approaching schools or districts to say, 'Look, we meet the evidence standard.' " Districts, she said, may not be able to easily check those claims. "They won't be doing an independent scan of what's available."
On the plus side, there is a big potential benefit to allowing districts to choose their own improvement strategies: They're more likely to stick with them, Wallin said. "I'm optimistic that people will choose interventions that are realistic for them to implement."
The capacity problem extends further up the food chain, too. It's unclear if states have the bandwidth to sort through scores of applications and gauge whether districts' proposed approaches meet the law's requirements.
What's more, there isn't a lot of detail in state ESSA plans on school improvement, experts who have reviewed the plans say. States seem to place a lot of emphasis on identifying struggling schools but not much on how they'll help them get better, said Mark Dynarski, a nonresident senior fellow at the Brookings Institution who reviewed a sample of 10 plans.
"The basic premise of the plans is that we are going to definitely know who these schools are and we're definitely going to get around to thinking about them as soon as we do," he said.
To be sure, the Education Department didn't ask states to put a lot of detail on school improvement in their ESSA applications. And Carissa Miller, the executive director of the Council of Chief State School Officers, argued that states are doing a number of things on school improvement that aren't necessarily outlined in their plans. State leaders feel a "moral obligation" to fix struggling schools, she said.
Some states are planning to guide districts through the turnaround maze by coming up with a list or database of evidence-backed strategies to choose from. Those include Connecticut, Delaware, Georgia, Illinois, Indiana, Iowa, Massachusetts, Minnesota, Nevada, New Mexico, New York, Ohio, Oklahoma, Rhode Island, and Washington, according to an analysis by Results for America's Evidence in Education Lab, a nonprofit that helps policymakers use research to inform decisions.
A couple of states—particularly Massachusetts—have been using research to guide improvement efforts for years.
But other states are putting new systems in place as they gear up for ESSA, including Nevada. Last year, the Silver State "sent out the Bat Signal" to organizations around the country that specialize in school improvement, said Brett Barley, the deputy state superintendent for student achievement. The state examined the strategies to see which ones met ESSA's standards.
And then Nevada had a "speed-dating event" where district and school leaders could interview organizations like New Leaders for New Schools to see how they could meet their needs. Jensen, for instance, chose to work with TNTP and the Achievement Network, two national nonprofits, as well as a local school improvement partner.
Finding high-quality providers on their own isn't something many district leaders in Nevada have time for, Barley said.
"Nevada is full of great educators, but I think the capacity issue is where we have seen things kind of fall down," Barley said. "We have superintendents who are also trying to figure out what the lunch schedule looks like and serving as the crossing guard." He said it's probably asking too much to ask them to also spend time examining the research behind different school improvement options.
Districts in Nevada don't have to choose a strategy from the state's list, but if they do, they'll get fast-track approval for their school improvement application.
The state is also using ESSA's evidence standards in distributing other federal and state funding so that districts can put money from different pots behind battle-tested strategies.
On the other side of the country, Rhode Island is taking a similar approach. The Ocean State is coming up with what state officials are calling a "hub"—a repository of strategies that have worked in different types of schools, with a range of challenges.
Districts that have their own improvement ideas don't have to rely on the hub. But those that are looking for evidence-based interventions and don't know where to start will have much of their work done for them, said Mary Ann Snider, the state's deputy commissioner.
"It's something beyond what you can accomplish in a Google search," Snider said.
Rhode Island is also asking districts with schools deemed among the worst in the state to set up "community advisory boards," which could be made up of parents, teachers, advocates, or anyone else the district thinks would bring a necessary perspective. These boards are expected to help conceive the district's school improvement plan and monitor the whole process, helping to make changes if students continue to fall behind.
The state is also hoping to set up partnerships between schools that have challenges in a particular area—like combating chronic absenteeism—and pairing them with schools that have come up with successful approaches.
But Scott of Education Northwest advises states against coming up with a "very prescriptive list of possible strategies" for school improvement. She suggests states and districts look to a national research repository, such as the What Works Clearinghouse, the Best Evidence Clearinghouse, and Results First. States can also partner with federal government's regional education labs, with universities, and with other experts to help make sure school improvement strategies have grounding in research, the CCSSO's Miller said.
'Chiefs Are on the Line'
Another looming challenge: Some states and districts searching for evidence-based strategies may have trouble finding something that meets their needs. For instance, there aren't many successful, research-backed ideas for fixing perennially foundering rural schools and those with big Native American populations, Scott said.
The field was aware of that challenge when ESSA was written, said Wallin, the former Education Department staffer. But supporters of the idea hoped that requirement for evidence-based solutions would spur more research into school improvement. "It's a little bit of a chicken-and-egg problem," she said.
Ultimately, state leaders have a lot riding on whether this new approach to school improvement can work.
"I think state chiefs are on the line to demonstrate that with this new autonomy they have they can move the needle," said Sara Kerr, the vice president of education policy implementation at Results for America, a nonprofit. "I think there's no way to do that without investing in approaches that are likeliest to result in improvement and studying the heck out of them and learning from them and iterating and improving over time."
Vol. 37, Issue 25, Pages 9-11Published in Print: April 4, 2018, as Districts Aim to Wield Evidence-Based Tools in Satisfying ESSA on School Turnarounds