Better evidence could mean more money for federal education and social programs, if a wide-scale interagency initiative proves successful.
Six new federal programs, including the U.S. Department of Education’s Investing in Innovation Fund, or i3, set aside different pots of money based on the level of research evidence that undergirds a project. The idea is to encourage developers to scale up proven programs and strategies while at the same time seeding research on less-tested ideas.
Developed as part of the 2009 fiscal-stimulus law, the i3 program has just received a strong application pool for its second, $150 million grant competition for fiscal 2011. (“Demand Strong for $150 Million in Latest ‘i3' Cash,” October 26, 2011.)
The new federal programs’ tiered levels of grants are intended to encourage developers to build a research case for social programs.
“There’s a need to build capacity both within the agencies and among the grantees about how to evaluate what constitutes good evidence,” said Jon Baron, president of the Coalition for Evidence-Based Policy. a Washington-based advocacy group.
“I think the Department of Education is playing a lead role in bringing other agencies up to that level,” he said.
In addition to i3, the grants include:
• The Workforce Innovation Fund, a $125 million program operated under the U.S. Department of Labor partnering with the Education Department to develop and scale up strategies to improve education and employment for workers;
• The Trade Adjustment Assistance Community College and Career Training Grants Program, a Labor Department initiative that is receiving $2 billion from 2011 to 2014 to create and expand education and career-training programs for dislocated workers;
• Money to the U.S. Department of Health and Human Services, totaling $1.5 billion from 2010 to 2014, to develop evidence-based home-visit programs for at-risk families with young children;
• Another $105 million to HHS in fiscal 2011 for the development of evidence-based, teen pregnancy-prevention programs; and
• The Social Innovation Fund, a $50 million pot of money under the Corporation for National and Community Service, to support public-private partnership investment in evidence-based programs in low-income communities.
Each program, like i3, provides larger grants to interventions that show strong evidence of their impact, up to and including randomized controlled trials, the so-called gold-standard in research. They also allot smaller grants to encourage practitioners to experiment with promising-but-untested interventions in ways that build in research evaluations from the start.
Seeding Partnerships
“Our old model was program dollars go out every few years … then we have research and evaluation dollars [in] a separate silo, and they’re all competing for resources,” said Kathy Stack, the deputy associate director for education and human resources at the federal Office of Management and Budget, in a presentation in Washington last month on the interagency initiative that led to the creation of the programs.
The six tiered-evidence grants are intended to encourage more direct partnerships between researchers and practitioners, Ms. Stack added, “to say, ‘You need each other, and we’re going to provide financial incentives for you to work together to learn, to build on existing evidence, learn what works, and produce new evidence that can support the growth of best practices and enable people to see what doesn’t work and find a graceful way of walking away from that to do the other stuff.”
Awarding grants based on different levels of evidence also helps “cut through tremendous politics” that can surround hot-button education and social issues, Ms. Stack said.
For example, the teenage-pregnancy-prevention grants in HHS are open to practitioners using both comprehensive and abstinence-only sex education programs.
“We basically said, ‘Hey, we’re going to be neutral. There’s room for everybody under this tent. You just have to demonstrate that you get better outcomes,’” Ms. Stack said. “That managed to calm down the politics.”
However, linking program evaluations to the size of grants awarded may encourage more people to game those research studies or focus on serving children who can improve with less help, said Robert C. Granger, the president of the William T. Grant Foundation in New York.
Moreover, increasing the rigor of evidence required for a program generally leads to fewer people meeting that bar; in i3, for example, only 46 of nearly 1,700 applicants received grants, which provoked some criticism.
“Because in this process there are more losers than winners, what you have is a political constituency for spreading the money around,” said Mr. Granger, who sits on the advisory board for the Institute of Education Sciences, the Education Department’s research arm.
That sort of congressional budget politics may shut down many of the programs before they have time to take root. House appropriators have cut evidence-based programs in HHS and Labor in the fiscal 2012 budget, though the Senate has so far protected them.
“The key to furthering this way of thinking—to steer money towards programs and policies that actually cause kids to do better in this world—is to make sure we learn things [from the research] that are useful to people who aren’t getting the money,” Mr. Granger said.
Even if the competitive programs don’t last, a September report for the United Kingdom’s National Endowment for Science, Technology and the Arts suggests the U.S. interagency initiative may represent “a new chapter in the generation and use of evidence by the federal government.”
Ron Haskins, a co-director of the Center on Children and Families at the Washington-based Brookings Institution and a co-author of the British report, said the competitive grant programs could be the tip of the iceberg, as all federal agencies are changing their grant-program criteria to require research evidence.
“This is a much broader strategy than the six evidence-based initiatives and if fully implemented could have a huge impact on federal social programs,” he said.
