In education, states and the federal government offer millions of dollars every year in free school meals, low-interest college loans, supplemental enrichment, and other supports that disadvantaged students often desperately need—but never take advantage of.
A year ago, the Obama administration called on federal agencies to dig into behavioral and social science research to figure out why the services go unused or programs aren’t as effective as they could be. Researchers and federal officials met at a White House symposium this afternoon to share what they’d learned from dozens of quick-turnaround evaluations and randomized controlled trials in agencies from Agriculture to Education to Housing and Urban Development.
Federal Agencies Experiment
For example, the Agriculture Department launched pilot programs to allow 20 states to use Medicare data to automatically enroll children in federal free and reduced-price meal programs over the next three years, after research in 11 states that already use Medicare data found that it significantly increased participation in the program, particularly among older students who may be reluctant to sign up for a program associated with poverty.
The White House social and behavioral sciences team is also creating and testing a web-based app that would let entire families apply for and verify information for school meal programs by emailing photos of documents rather than having to mail them or deliver them by hand. Prior research found up to half of students who lose access to school meals for not verifying their eligibility did in fact qualify for the benefits.
Similarly, Calvin Johnson, HUD’s deputy assistant secretary for research, said his agency is testing different versions of mail and electronic messages to students in public housing, educating them about ways to apply for federal college loans. So far, nine different versions of a mailed message have produced only small increases, but the group did learn details about the most effective formats and content styles to encourage students to apply for federal financial aid.
New Federal Guidance on Research and Evidence Coming
The Education Department plans to release new guidance on using evidence under ESSA tomorrow. We’ll have more on the details then.
Ron Haskins, a co-chair of the new federal Commission on Evidence-Based Policymaking and the co-director of the Center on Children and Families at the Brookings Institution, said the federal work to date has highlighted future needs for evidence-based policymaking as districts work to implement the Every Student Succeeds Act’s evidence standards for school improvement:
- Agencies need more creative interventions. “Our interventions stink,” Haskins said. “We need very creative program operators and then to work with evaluators to evaluate as they roll them out.”
- Narrow the core in programs. “We can’t completely replicate programs,” Haskins said. Program developers should “identify four or five key ingredients to replicate” that will allow districts to adapt programs for local contexts without losing what makes a program effective.
- “Perform more and more evaluations.” State officials and and local university researchers should help “sell” local districts on the need to evaluate their programs, Haskins said. “The goal should be to make sure very high-quality evaluation capacity is available to local programs.”
- Get used to failure. Haskins and several others at the conference noted that about 80 percent of rigorous evaluations don’t find evidence of effectiveness. Districts will need to learn from these failures, rather than becoming averse to studying their programs because of them. “University people could play a very important role for making evaluations pretty much mandatory for any program that wants to be on the map,” he said.
Related: