Educators have a lot of questions about AI. As do I. Well, when I need practical takes on ed tech, I turn to the ever-thoughtful Michael Horn—lecturer at the Harvard Graduate School of Education, co-founder of the Christensen Institute, co-host of the Future U podcast, and author of many books, including Blended: Using Disruptive Innovation to Improve Schools. As school leaders weigh a new crop of AI tutors and math programs, I asked Michael why the results appear to be so uneven—and what can be done about that. Here’s what he had to say.
—Rick

Rick: Michael, school leaders are being bombarded by shiny AI-powered tutoring models. They all seem promising. Then they get adopted, and the results disappoint. The big, recurring problem is that only a small share of students use them as recommended. Laurence Holt has dubbed this the 5% problem. These wares may be great for the students who use them, but most just don’t seem inclined to do so (which sounds suspiciously like what happened with MOOCs 15 years ago).

You’ve been writing about this dynamic for years. So, what’s going on? And what’s this mean for AI-backed math programs that aren’t “tutoring” so much as the backbone of math instruction?

Michael: Rick, put yourself in the shoes of a teenager. Do you want to do hard work if there’s no clear connection between the effort and an outcome you actually care about? I mean, how many college students even do all their assigned reading? It seems like you have to keep it under 30 pages a week to have a chance these days—and even then, it’s probably a long shot.

As your AEI colleague Nat Malkus has shown, many schools are still struggling just to get students to show up consistently—and we somehow expect those same students to voluntarily fire up an AI-powered tutor with no human accountability attached to it?

It comes back to what the psychologist Daniel Willingham has long argued: Our minds are wired to take the path of least resistance. Given the chance to avoid hard work, most of us take it. So, while a lot of smart people think AI tutors are getting better and better, unless we start paying real attention to motivation and social accountability, I wouldn’t expect much usage or much benefit.

Rick: OK, that makes sense. So, if an AI-powered tutor purportedly delivers big gains, what questions should educators or parents be asking? Is combating the “5% problem” as simple as making sure students are tested and held accountable for the results?

Michael: These are big questions, Rick.

First off, we too often start with the technology and discount the learning model itself. Technology isn’t a silver bullet in education. Crammed into the existing classroom model, it can at best serve as an add-on that bolsters the processes and priorities already in place. It can make a classroom more efficient, but it can’t reinvent the classroom. And part of the problem is that the technology often conflicts with a traditional classroom’s existing routines—so it ends up going largely unused.

Here’s one conclusion: Before focusing on the technology, we need to redesign the learning model itself. Imagine a basic station-rotation model of blended learning, where students get an uninterrupted block of time to work on a digital math program—or, in this case, to get support from an AI tutor on the work they need to complete. Right away, every student reaches minimum usage levels.

Rick: Of course, as you frequently note, integrating technology into learning is different from getting students to use those tools effectively. What does it take to address that second challenge? And what does that demand of those designing and deploying these tools?

Michael: To bastardize a line you often use about education policy: We can require students to use technology, but we can’t require that they use it well. Sure, a kid might be sitting in front of a computer. But are they engaged and are they working on what they’re supposed to be in order to learn?

Getting that right isn’t an accident. The classroom models educators teach in and the tools engineers build both have to be designed with motivation in mind. Two things follow. First, students have to be accountable for actually mastering the material, not just logging time. Second, a model needs a coherent theory of motivation baked into it, rather than bolted on as an afterthought.

For example, say what you will about Alpha School, but at least they have a clear theory of the case: Do two hours of hard academic work in the morning, and the afternoon opens up for the fun stuff. I personally think this model neglects the fact that hard academic work can be rewarding if designed well. But at least they have some real coherence and accountability.

For everyone else, a few lessons worth considering. Connect new learning to the background knowledge students already have, so the work feels within reach. Reward the hard grind of mastery with a tangible sense of progress, so students can see the ground they’ve covered. And when a student is on a device, lock down everything except the AI tutor they’re supposed to be using. Get rid of the distractions.

This conversation has been edited for length and clarity.