The bipartisan Commission on Evidence-Based Policymaking is set to release its final report in September, but Shelly Wilkie Martinez, the commission’s executive director, hit some of its highlights in a keynote Tuesday for the National Center for Education Statistics’ annual data forum.
Congress charged the $3 million,18-month commission to come up with ways to coordinate research and data use across federal agencies while protecting data privacy and security. The commission is not focused just on education, but it is likely to shape reauthorization of critical and long-overdue education laws governing research and data—including the Family Educational Rights and Privacy Act of 1974 and the Education Sciences Reform Act of 2002.
“I don’t think this is a spoiler alert but I think we’re going to need some changes in laws and policies to make all of this work,” Martinez said.
The group has held seven meetings, and three hearings with everyone from state and academic data experts, to parent and citizen privacy advocates, to directors of data systems in other countries. It also studied data and evidence creation and use at more than 200 federal agencies.
It’s been months of digging in a field full of landmines. For example, Martinez said that the week after many researchers and postsecondary education groups argued to end a ban on federal “student unit data systems” which would track individual students’ data from K-12 schooling through college and into the workplace, the commission was “flooded” with arguments from parents and privacy groups along the lines of “No, never, not at all, definitely not!”
“There’s a lot of passion around student privacy,” she said.
While the commission supports high-quality research to support programs, Martinez did not put a dog in the fight over how to balance qualitative versus quantitative research. “We have a recognition that there are a lot of different camps that advocate a particular method, and we have resisted doing that,” she said.
While she did not dig into the details of the commission’s report, she outlined five key principles that members agreed will need to govern the future of research and data use:
- Privacy: “As we think about the creation of new evidence through access to data, privacy has to be in the forefront of our minds,” Martinez said.
- Rigor: “There’s a lot of different kinds of evidence and evidence-building that are essential. We heard some comments that there are certain kinds of methods that are more rigorous than others, and the commission really wanted to reject the idea that some methods done well are, by nature, less rigorous. ... Regardless of the method, you should use a method that fits the question with an appropriate level of rigor. The idea of a portfolio of methods emerged for us, that we need different methods to get at answers to a policy question.”
- Transparency: “In the days when primary data collection was the main way of building evidence, it was really clear who was asking for data and for what, but in an era of more secondary data collection, we lose some of that connection to the people whose data we are using. ... So the idea of being transparent struck us as very important.”
- Humility: “It begins to really call into question the entire enterprise of science when people feel like the next study down the pike is going to have a very different finding.” She added that agencies should avoid overgeneralizing findings and instead focus on building and promoting a broader “portfolio of evidence.”
- Capacity: “None of these privacy protections and data uses can happen without a lot of expertise.”
“People are hoping this is a milestone moment—that we will come out of this process with legislative recommendations, executive branch recommendations, recommendations for other actors in our ecosystem that we hope will launch us in a new direction to begin to build a more systemic approach to building evidence that relies less on heroic individuals and becomes more a routine government operation,” Martinez said.
Check out this explainer for more on how the Commission works.