As regular readers know, I’ve long held that American education is overdue for a rethink but also that most “reform” is oversold. Finding the sweet spot, as Michael Horn, Julie Squire, and I note in our new book, School Rethink 2.0, requires focusing on rubber-meets-road change. Rethink 2.0 offers accounts from leaders doing that work, and I’m delighted to speak with one of them today. Kristen DiCerbo is the chief learning officer at the acclaimed Khan Academy, where she oversaw development of Khan’s AI-powered tutor Khanmigo and was named to Time Magazine’s AI 100 in 2024. Khanmigo has been described as a “personal tutor and teaching assistant” and is designed to support users with mathematics, science, humanities, and coding questions. Here’s what she had to say.
—Rick
Rick: In School Rethink 2.0, you write about developing the Khan Academy’s AI tutor Khanmigo. How did that come about?
Kristen: Our initial impetus was a 2022 meeting where leadership from OpenAI demonstrated a new model they had just finished training, which we now know as GPT-4, and gave us the chance to interact with it. After a few days of doing all the things everyone probably did when they first got access to a large language model—writing poems for our partners and asking it all kinds of questions—we started thinking about whether it could help address some of the persistent problems we have in education. We decided that it could have a significant impact on the ways students get support as they learn, so we threw away our road map for what we thought we would be working on and started building Khanmigo.
Rick: So you quietly started designing Khanmigo even before ChatGPT was released. What was it that grabbed your imagination?
Kristen: Our first lesson in prompting was what initially amazed me. We could provide it with some basic instructions, in English rather than code, and change the kinds of output it gave. If we said, “You are a Socratic tutor. I am a student. Don’t give me answers to my questions but lead me to get to them myself,” it returned different responses from what it returned when we asked it a question without those instructions. It started doing some of the things good tutors do. We spent September and October 2022 deciding what we were going to build and doing basic prototyping. We then started seriously designing and building in November and launched in March 2023, so it took seven months to develop Khanmigo.
Rick: What were some early lessons you learned while designing Khanmigo?
Kristen: Consistently getting the AI to do what we wanted was not as easy as giving it three sentences of instructions! Even when given the same prompt and set of instructions, the model provides a different answer every time. You can’t just test a prompt once and call it a day. We had to set up evaluation systems to repeatedly run each prompt and check the results of those tests. Then, we had to keep track of the most recent version of each prompt since we were constantly tweaking the language. We also learned that when prompts get to be too long, the model starts randomly ignoring different parts of the instructions. What kept us motivated was remembering our core goal: giving students better support and feedback as they practice.
Rick: Have there been any surprises about how educators or students are using Khanmigo?
Kristen: I have been designing and researching educational technology for 20 years, and the first thing that surprised me was how quickly AI has been adopted. We went from about 68,000 Khanmigo student and teacher users in our partner school districts in 2023-24 to more than 700,000 in the 2024-25 school year, expanding from 45 to more than 380 district partners. In total, Khan Academy partners with around 450 districts in the U.S., meaning that most of our partner districts have decided to use Khanmigo. I was also surprised by the number of students and teachers raving about Khanmigo as a tool for English-language learners to get support in their native language since it can answer students’ questions in many languages. I was also surprised by the number of teachers using Khanmigo to refresh their own knowledge. However, I’m a little disheartened by how often some teachers would primarily use our AI to generate multiple-choice questions. I’d love to see students engage much more deeply with content through AI, something that multiple-choice questions rarely encourage. I would like to see greater usage of Khanmigo activities like Ignite My Curiosity, which allows students to engage in conversations around interesting ideas in domains they may not be familiar with.
Rick: As you know, there are concerns that students are spending too much time on devices. What’s that mean for AI tutoring?
Kristen: Prior to generative AI, our efficacy research showed that students who used Khan Academy for an average of 30 minutes of additional math practice per week throughout the school year saw greater-than-expected gains on standardized assessments. Incorporating AI is not meant to add even more time on top of that but to make students even more productive with the time they have by providing active support and feedback during study sessions. We in no way envision a classroom where students spend all of their time individually working on screens.
Rick: What are some of the practical challenges when it comes to AI tutoring?
Kristen: The biggest challenge we are facing with Khanmigo is the same challenge we have seen historically with educational technology: achieving meaningful student engagement. We know tools like Khan Academy will work … if students use them correctly. When I review student chats with Khanmigo, I see some conversations where students are doing exactly what we would hope by answering questions and then posing their own to deepen their understanding. I also see a lot of chats where students are responding, “I don’t know,” or, my recent favorite, “Bro, IDK.” If students are not putting forth cognitive effort in their interactions, there is no reason to expect that they will learn from them. In some cases, we also see that students don’t actually have the skills to recognize what they know, what they need to get support on, and how to write good questions to address their weaknesses.
Rick: What helps students figure out the right questions to ask?
Kristen: The teacher is the most powerful tool to address this. For example, one science teacher worked with her students to come up with good question stems to start questions for Khanmigo. She then printed them out on 3x5 cards to put on their desks. It was quite a surprise to enter the classroom and witness students referencing cards on their desks, just like I had in the 1980s, to teach them how to interact with this new technology. The broader lesson is that students and teachers need the skills and the motivation to use AI well.
Rick: What do people get wrong about AI and tutoring?
Kristen: AI doesn’t change the fundamentals of how kids learn. When we learn new things, we need practice with support and feedback to master them. In too many cases, students just don’t get enough opportunities, whether in math, writing, or other domains, to effectively practice these skills. AI tutoring makes it easier for every student to get that practice.
Rick: AI is developing at a breakneck pace. What does that mean for your efforts?
Kristen: In the near term, Sal Khan is still on the hook for a good number of videos for new courses we’re launching. We have found that AI dubbing of his voice into other languages has gotten very good. We will be launching that soon, with appropriate labeling. We are continuing to work on bringing new AI technology to the tutoring experience. For instance, we just launched image inputs. Now, Khanmigo can interact with students about an image they share. This is critical for domains that rely on visual representation, like geometry. We are inching closer to the place where the model can collaborate in real-time with students about work done on a scratchpad. Finally, we’re interested in how AI might impact assessment, ranging from developing more authentic kinds of assessment activities to helping teachers, parents, and students better understand their scores.
Rick: As educators get more comfortable with AI, do you have any advice on how to best ensure they’re making sound decisions for learners?
Kristen: Always focus on what problems you are trying to solve, not the technology you have available. Then, ask whether the technology will help solve that problem, and if so, how it can help. Finally, develop a method for measuring progress—ask how you will know if that new technology is working. We are in the early days of the journey with AI in classrooms, so there isn’t a lot of high-quality data to rely on. This means teachers and schools may want to start slowly and gather their own evidence about what works with AI. But I would echo advice I heard Rebecca Winthrop of the Brookings Institution give recently: Don’t let FOMO drive your decisions.
