State education officials are open to using research to shape policy and practice decisions, but they say that it remains difficult to make practical use of most studies.
Margaret E. Goertz, co-director of the Consortium for Policy Research in Education at the University of Pennsylvania, probed the research use and practices of three states in 2010 and 2011. At an American Youth Policy Forum briefing on Capital Hill this afternoon, Goertz and other experts discussed ways to help researchers and state policymakers get on the same page.
Goertz found that staff in all three states relied the most heavily on research generated in-house, or on advice from other colleagues; more than 70 percent of state education officials consulted colleagues about research at least once a week. A third to half of them also used published original studies and summaries and research-based guidance, particularly from federal sources.
Even though policymakers and program officers in the states were open to research, they also said they need considerably more support from so-called “knowledge brokers,” who can boil down research findings and translate them for the specific state’s context.
Researchers themselves generally don’t take the extra step of ensuring their findings are relevant and usable, according to Carrie Conaway, Massachusetts’ associate education commissioner for planning, research, and delivery systems. “Researchers need to take a class on writing for a policymaker audience rather than for a research audience,” Conaway said. “Please, please write in English! So many times I look at it and wish for a graph, just a simple graph that anyone can understand, because it’s a sea of numbers.”
Goertz found state education officials were least likely to use research evaluating programs and interventions—not least because they found high-quality program evaluations thin on the ground (An opinion I’m sure made the What Works Clearinghouse staff wince).
Massachusetts officials would have made frequent use of evaluation and implementation studies “if they had existed,” noted Conaway. “Implementation studies would have saved us a lot of time reinventing the wheel.”
The Council of Chief State School Officers has been trying to help states make better use of each others’ research and data through its Innovation Lab Network, a 10-state initiative to develop research networks around policy questions and share both research and implementation differences that might affect how findings apply in different states.
The Innovation Lab Network is based on what interim director Jennifer Davis calls the “ASSETS” method, in which states and researcher partner to:
• Aggregate problems of practice to identify questions to research;
• Survey existing research and identify gaps;
• Sketch research design;
• Enact the research study;
• Tally and communicate results; and
• Simulate lessons learned based on the results.
It’s that last part that’s both intriguing and tricky. Davis said communicating policy research has to include laying out the trade-offs made along the way, the decisions made that led practitioners to choose a certain approach. Doing so allows policymakers in other states to understand whether and how to try the same approach in their own states and may cut down on the frequent complaint that highly successful new programs and policies fall flat when taken to scale.
Conaway also echoed Goertz’s call for federal and state legislators to require—and reserve funding for—evaluation in all programs from the outset, rather than relying on state or local program officers to decide to study the programs later on.
“Given the choice between spending money on a program and spending money on an evaluation of a program, they will spend money on programs every time,” Conaway said.
A version of this news article first appeared in the Inside School Research blog.