Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Federal Opinion

‘Jargon’ and ‘Fads': Departing IES Chief on State of Ed. Research

Education isn’t living up to its potential. Its researchers aren’t, either.
By Rick Hess — April 08, 2024 7 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

Just last month, Mark Schneider wrapped up his six-year term as the director of the Institute of Education Sciences. At IES, he was charged with overseeing the nation’s education research efforts, including such well-known efforts as the National Assessment of Educational Progress and the What Works Clearinghouse. Before assuming his role at IES, Mark was a vice president at the American Institutes for Research, commissioner at the National Center for Education Statistics, and spent many years chairing the political science department at SUNY-Stony Brook. Having known Mark for many, many years, I was interested in his reflections on his tenure at IES. Here’s what he had to say. (This is Part One of a two-part interview, the second of which is scheduled to be published on Wednesday.)

—Rick

Rick: You’ve just wrapped up your tenure as director of the Institute of Education Sciences. What’s the state of American educational research?

Mark: When I was a political science professor at Stony Brook University, fights over resources often pitted the “hard sciences” against the social sciences. At faculty meetings, I would use James March’s observation that “God gave all the easy problems to the physicists.” That would lead to plenty of eye-rolling from everyone but the social science chairs and faculty. But the fact is that humans have far more agency than electrons, and so many of our most beautiful and parsimonious models crash and burn when tested against real-world data. Our work is difficult, and we need to face the reality that most of our ideas, hypotheses, and interventions will not stand up to empirical reality. Indeed, IES-supported interventions have a “hit rate” that hovers around 15 percent, meaning most things don’t work. Depressing, but that success rate is not much different from most other human-centered fields. Indeed, by some estimates, only 10 percent of clinical trials succeed.

Over its 20-year history, IES has developed and perfected a standard “business model.” I summarize it as: “Five years, five million dollars, failure,” which reflects our standard grant length, funding level, and outcome. This may sound like an indictment of how IES has supported the education research “industry,” but we need to understand the reality we face. And we need to embrace the idea that “the only failure in failure is the failure to learn.”

Rick: Given the challenges you’ve just sketched, how have you sought to shift IES efforts?

Mark: I have tried to push IES to change its standard business model. Most notably, there are now a growing number of large learning platforms—IES’s SEERNet among them—that let researchers access large numbers of students, enable rapid-cycle experiments, and support more frequent replications. IES’s mission is to determine “what works for whom under what conditions.” This can only be achieved by replicating work in different audiences as a routine part of our research. Of course, other sciences have supported replications for years. Many now face a “replication crisis,” in which other research teams—or even the research team that conducted the original study—cannot verify findings. We have not been pursuing replication work long enough to generate a replication crisis in our field, but I look forward to that day.

The field is also beginning to face the problem that even after we identify practices and programs that “work,” we do not scale those interventions by moving from, say, 200 students in an experiment to 2,000 students in a school district to 2 million students in the nation. Without paying far more attention to scaling, we will never affect enough students to achieve our goal of creating a strong democratic citizenry capable of earning family-sustaining wages.

Rick: As director, what’s been your biggest frustration with the education research community?

Mark: Education science, like many disciplines, is overly affected by jargon and fads. We are an applied science charged by law with helping legislators, educators, parents, and students understand what we are saying and how to join us in implementing changes. But we often fail to translate our work into plain, accessible English. I have tried to impose several policies to make IES’s publications and reports more user-friendly—for instance, by imposing a 15-page limit on our reports. But perhaps the biggest indictment of the field is that the writing in education research reports, including IES’s own, is often amazingly poor.

“Short sentences. Strong verbs.” That’s the approach I ask people to bring to their writing, yet we rarely hit the mark. Perhaps the problem is that so many IES staff and contractors have Ph.D.s, so they think a greater number of poorly crafted reports is better than fewer tightly focused ones. As a result, our journals drip with incomprehensible jargon and big words that signify little.

Rick: What kinds of changes could help on this count?

Mark: For several years, I have been pushing for the creation of a new center in IES focused on informed risk, high reward, and rapid-cycle research. Like the establishment of centers throughout the federal government modeled on the Defense Advanced Research Projects Agency, or DARPA, the new IES center—currently called the National Center for Advanced Development in Education, or NCADE—would breathe life into a large segment of IES’s research expenditures. The creation of NCADE would require an act of Congress.

We also need to continue updating the SEER principles. The Standards for Excellence in Education Research lay out some best practices that would guide the education science field to produce stronger work, plus some principles that are specific to education science research. Should the next director continue to refine these, I believe that our field will be far better off than it was earlier in IES’s life—and even more so when comparing the quality and the rigor of today’s work with that of the pre-2002, pre-IES world.

Rick: You argue eloquently for new research centers within IES. But IES already has four of them. What makes you confident that a new one, NCADE, would fare any better? Put another way, why can’t the existing centers just do the research that you’d ask NCADE to do?

Mark: This is the hardest question to grapple with when considering the need for NCADE. I sometimes think that if COVID hadn’t punched a two-year hole in my tenure, I might have been able to make enough changes in the culture and the practices of IES’s two existing research centers to alleviate the need for NCADE. But changes in personnel, practices, and especially culture in a federal bureaucracy, even a relatively small one such as IES, are among the most persistent challenges any reform-minded leader faces.

I can document many barriers that are just hard to overcome. It is not surprising that a time-honored solution to such inertia is to create a new bureaucracy in which modern practices and new personnel can be brought in to drive new initiatives. On an abstract level, do we really need NCADE? Maybe not. But if we want to modernize and streamline education science research, NCADE is, I believe, the best and fastest way to bring about changes the field—and the nation—needs.

Rick: You’ve made a priority of accelerating the speed at which we release federal education data and research. What’s the problem and what’s holding up efforts to do better?

Mark: The lack of timeliness is the single most common complaint that IES fields year after year. Many of these complaints come directly from Congress. Indeed, when the Senate HELP Committee drafted a reauthorization of the Education Sciences Reform Act, the senators added a new paragraph about the responsibilities of each commissioner with the same title: “timeliness.” Our research centers, for instance, are largely focused on measuring how students grow and learn—which can take years. On top of that, we often fail to push for timely publications and product development once that research is complete.

Meanwhile, IES’s National Center for Education Statistics, or NCES, has other problems that slow down the release of its work. Officials at NCES argue that they simply don’t have enough staff to produce timely products. And maybe they don’t. But they also do not seem to understand their audience’s need for access to fast and reliable information.

Researchers like Marguerite Roza, Emily Oster, and Sean Reardon are helping to fill in the gap by standing up work that NCES could and should do. The signature example is how slow NCES has been in releasing critically needed information on how schools and districts are spending money. There will always be a lag between data collection after a fiscal year ends until it can be released—but the lag has often been years rather than months. I don’t think that’s a result of limited staffing. I think it’s a result of limited vision.

Part of my push for NCADE has been to break the mindset of too many IES staffers and to bring in new blood committed to identifying critical shortages in education research and pushing for more rapid solutions, including greater sensitivity to the needs of research consumers—which would almost always mean faster turnaround.

Related Tags:

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

School Climate & Safety K-12 Essentials Forum Strengthen Students’ Connections to School
Join this free event to learn how schools are creating the space for students to form strong bonds with each other and trusted adults.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Creating Confident Readers: Why Differentiated Instruction is Equitable Instruction
Join us as we break down how differentiated instruction can advance your school’s literacy and equity goals.
Content provided by Lexia Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
Future-Proofing Your School's Tech Ecosystem: Strategies for Asset Tracking, Sustainability, and Budget Optimization
Gain actionable insights into effective asset management, budget optimization, and sustainable IT practices.
Content provided by Follett Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Miguel Cardona in the Hot Seat: 4 Takeaways From a Contentious House Hearing
FAFSA, rising antisemitism, and Title IX dominated questioning at a U.S. House hearing with Education Secretary Miguel Cardona.
6 min read
Secretary of Education Miguel Cardona testifies during a House Committee on Education and Workforce hearing on Capitol Hill, Tuesday, May 7, 2024, in Washington.
U.S. Secretary of Education Miguel Cardona testifies during a House Committee on Education and Workforce hearing on Capitol Hill on May 7 in Washington.
Mariam Zuhaib/AP
Federal Arming Teachers Could Cause 'Accidents and More Tragedy,' Miguel Cardona Says
"This is not in my opinion a smart option,” the education secretary said at an EdWeek event.
4 min read
U.S. Secretary of Education Miguel Cardona speaks during Education Week’s 2024 Leadership Symposium at the Hyatt Regency Crystal City in Arlington, Va., on May 2, 2024.
U.S. Secretary of Education Miguel Cardona speaks during Education Week’s 2024 Leadership Symposium at the Hyatt Regency Crystal City in Arlington, Va., on May 2, 2024.
Sam Mallon/Education Week
Federal Opinion Should Migrant Families Pay Tuition for Public School?
The answer must reflect an outlook that is pro-immigration, pro-compassion, and pro-law and order, writes Michael J. Petrilli.
Michael J. Petrilli
4 min read
Image of a pencil holder filled with a variety of colored pencils that match the background with international flags.
Laura Baker/Education Week via Canva
Federal New Title IX Rule Could Actually Simplify Some Things for Districts, Lawyers Say
School districts could field more harassment complaints, but they can streamline how they handle them, according to legal experts.
7 min read
Illustration of checklist.
F. Sheehan for Education Week + iStock / Getty Images Plus