It’s been a fascinating few months since David Coffey and John Golden posted Mystery Teacher Theatre 2000 on June 18. Since that time, their video has been viewed nearly 35,000 times, nearly 30 other #mtt2k videos have been created, and there have been dozens of articles and posts in venues from Fast Company to the Washington Post. I’ve personally learned a ton about math education, the math twittoblogosphere, Khan Academy, the power of satire, the passion of Khan aficionados and critics, and the unconcionable dearth of talking robots available for educational work.
Here is where the social studies teacher in me comes out: there have been a variety of thought-provoking conversations about Khan Academy sparked by David and John’s video (and nudged by this contest). For everyone who used these conversations to think more about Khan Academy and to share your thoughts, thanks for expressing your ideas and pushing mine.
So everyone is entitled to their own takeaways from the dialogue, but here are my big learnings. I may post down the line about the dynamics of the Khanversation (satire, trolls, puns with Khan’s last name, etc.), but for now, I’m going to stick to math, technology and education.
We were Promised Jetpacks and Got Lectures
For me, the primary lament of #MTT2K was this: We’re in a revolutionary age of low-cost media production and sharing, Gates and Google have given Khan Academy millions and millions of dollars, and what they have to show for it is scaled-up, online versions of lectures, worksheets, and stickers. Dammit, we were promised jetpacks! The media said that this was the future of education.
In its present form, Khan Academy accepts the premise that the fundamental building blocks of math education are 1) didactic lectures to teach math, 2) worksheets to demonstrate proficiency and evaluate students, 3)stickers (leaves/stars/points) to motivate students, and 4) teachers to act as coaches when students get stuck. Khan Academy rearranges those pieces without fundamentally changing their nature. Khan Academy’s primary move is to make the lectures, worksheets, and sticker-distribution mechanisms freely-available online, so that students can move through a standardized curriculum at an individual pace, and so teachers have more time to tutor (Frank Noschese’s video, Khan Academy Classsrooms: Rhetoric and Reality, effectively illustrates what these classrooms now look like). Everyone, however, immediately recognizes the lectures, worksheets and stickers as riffs on common elements of math education. In fact, part of the success of Khan Academy is that these unchanged core elements are immediately recognizable to everyone.
So, the question is, does scaling up the delivery of well-establish products constitute a revolution? Did the Gideons produce a revolution in Bibles by leaving them in lots of hotel rooms? Jetpacks would be a revolution. Talking robots would be a revolution. Are a lot of video lectures on math, with a set of online practice problems, a revolution?
This is a genuine question and one of the lynch pins of how critics and aficionados frame Khan Academy.
This question is also at the heart of Michael Pershan’s Grand Prize video “What would Khan Academy Look Like in Japan?” and Dan Meyer’s own “Khan Does Angry Birds.” Michael’s framing is that we have strong evidence from numerous countries with successful math education programs that great mathematics instruction starts with questions rather than instruction in procedures. His argument is that Khan is scaling up a set of math education building blocks, but we have strong evidence that this set isn’t the best set of fundamental math education forms that humans have developed.
Dan’s research and career at this point is devoted to determining how we can create new fundamental building blocks of math education, ones that develop problem-solving dispositions, the ability to identify relevant information, prediction skills, hypothesis testing, and other skills along with math algorithms. He’s developing a set of building blocks that engage students’ inborn curiosity about the world around them and our inclination towards organizing in narrative structures, with questions, conflicts, tensions, and resolutions. If you haven’t yet, go check out Dan’s Three-Act math materials, and the ideas that he’s been riffing on in his recent blog series on the Ladder of Abstraction. You’ll see a bunch of stuff that looks unfamiliar to the public and to many math educators. He’s not thinking about using technology to scale our current math education, but to create a new fundamental building blocks of math education.
I do think it’s incredibly important to engage this question of whether or not Khan Academy constitutes a revolution in math education, because it’s a powerful case for asking one of the fundamental questions in education today: should we be using technology to do old things more efficiently, or to do new things? A lot of the fawning media coverage of Khan Academy doesn’t necessarily advance this discussion; in my mind, that’s why #MTT2K was most needed.
On the Wisdom and Experience of Math Educators
A second line of critique is one well articulated by Chris Danielson and Michael Goldenberg in their post at Valerie Strauss’s blog in the Washington Post: Sal knows the math but makes errors in math pedagogy that are common to new math educators. Danielson and Goldenberg use Lee Shulman’s notion of Pedagogical Content Knowledge to help make their argument. Shulman’s theory of posited three forms of knowledge useful for teaching: pedagogical knowledge (general ideas of how to teach), content knowledge (expert knowledge from the field), and pedagogical content knowledge, which is knowledge specific to instruction in a particular subject area.
The original MTT2K video hit on this point several times. For instance, Coffey and Golden observed that Khan was pretty loose about labeling positive numbers with a “plus” sign and interchangeably calling them “plus seven” and “positive seven.” For experts in math, the imprecision in terminology is irrelevant. But for new math learners, this kind of imprecision leads to misconceptions.
Along the same lines, Debbie Borkovitz did a lovely analysis of a video on fractions with a litany of observations about Khan’s pedagogical moves that expert math educators recognize as pathways to misconceptions. My favorite example: pies and pizzas are over-represented in fraction examples, leading some students to equate the notion of fractions with the concept of area. Borkvitz recommends using a variety of examples including volume (gas tanks) or distance. She also noted that one common confusion around fractions is that students think they represent 2 numbers rather than 1 number, and that “unitizing” fractions for students is quite important. This isn’t nitpicking; these insights can dramatically improve direct instruction in these subjects.
Joe Kremer had a running analysis of a Khan video on acceleration, which definitely reminded me of some of the teaching observations I’ve done where you realize early on that the teacher is headed for the weeds. The lecture is ostensibly about acceleration, but Khan selects an example requiring some extensive unit conversions (miles per hour into feet per second). That might be a useful video for “How to solve acceleration problems that you are likely to find on tests,” but a more experienced teacher would recognize that an introduction to a complex new concept is not the place to get lost in a quagmire of unit conversion. Khan does the problem right, it’s just the wrong problem for the circumstance.
The math educator community has within it an incredible wealth of nuanced understanding of the kinds of misconceptions that students run into and the kinds of instructional errors that novice teachers make in the early days of their instruction. One of the great weaknesses of the Khan enterprise is that Sal doesn’t have the opportunity to follow students along and to see where his best pedagogical moves lead to blossoming wisdom and to see where his mistakes lead to student confusion and misconception. For me, the #MTT2K videos highlighted just how much math educators know about their craft, and it opened a window for me to see some of the places where Khan’s lectures make these well-understood pedagogical mistakes.
Now, one important point is that developing these nuanced teaching strategies takes time. So much time in fact, that if Khan had spent his hours studying math pedagogy, maybe he wouldn’t have time to scale up 3,000 videos. There is a value to getting the whole curriculum up online in a relatively short period of time. And there really is something to be said for the idea, “don’t listen to the experts and the old guard, do something new and innovative.” But, certainly I’m hopeful that one of the results of #MTT2K is that Khan will look back through his oeuvre with some experienced math educators to guide him, and look at places where the lectures can be improved.
A less compelling defense of Khan’s pedagogy is “lots of people like these videos” (187 million lessons served), so they must be good. One simple retort is that quantity or popularity, doesn’t equal quality. Dr. Tae has a funny riff on that where he pretends to be an expert pool player (QUOTE), and Audrey Watters notes the obvious comparison to MacDonald’s billions and billions served. A more nuanced point comes from Derek Muller and his research on science videos. You can watch a summary of his argument here, but the nub is he ran a nifty study where Muller showed people videos with simple explanations of phenomena and more complicated discussions of the misconceptions of those phenomena. People liked the simple videos better, and learned nothing from them. People thought the complicated videos were more confusing, liked them less, and learned more from them. In other words, we might expect certain kinds of inverse correlations between simple, digestible, popular videos, and ones that effectively instruct. Ben Kamens has written some cool posts about how Khan Academy can use A/B testing to potentially test the impact of changes in Khan videos to answers to practice problems, but I’m not sure that the current crop of practice problems are the right instrument to test deeper conceptual understanding.
On the Future of Khan Academy
My final point is one that Khan makes: this is not a finished product. As he wrote me “We think we are just at the beginning stages in our quest to make a truly valuable resource for the world and are eager to improve as much as possible (especially our older videos originally made for my cousins).” I really do believe that the Khan team is driven to constant improvement. In fact, much of this conversation started because Khan made four videos in response to the original MTT2K video. I hope that listening to experienced math educators is a big part of their feedback processes. I hope that considering the limitations of the building blocks that Khan is currently scaling up is another. I’m really excited to see where Khan Academy grows from here, and it is an enterprise brimming with potential.
When I started the contest, I repeated my claim that I can’t see any reason why every kid in America shouldn’t have a Khan Academy account. I still stand by that claim. For the right kid at the right time, Khan Academy could support a struggling student or provide a superior education to the one available in some cases. It’s a good contribution to the planet. But like any pedagogical resource, educators, parents, and students need to scrutinize it carefully.
I also repeated my claim that I think Salman Khan is a truly philanthropic spirit, and I continue to stand by that. He could have sold Khan Academy for a big pile of money to Pearson, McGraw-Hill, or whoever, but he didn’t. He drew a $70,000 salary in 2010 when Khan Academy had nearly two million dollars in revenue and contributions. He’s an advocate of openness and of expanding opportunity. I’m planning on inviting him to visit or Skype into my education class for MIT undergrads this spring, and I hope my students will follow his footsteps in bringing their considerable talents to the education field. I think we can and should have a very robust debate about the design and impact of Khan Academy, and it’s vital that educators and the public critique him as a lecturer—personal attacks on his motives or character are, to my mind, beyond the pale.
I’ve had a great summer thinking with the #MTT2K community about Khan Academy, and on this last day of August, I’m pleased to be done for a while. Much thanks to Sal and co for his contributions to education, which were significant enough to merit a critical discussion; much thanks to the whole #MTT2K crew for a summer of rich learning. To all those who are getting their kids logged into the Khan Academy this September, to all those who are done with Khan forever, to all those out there trying to build something better (including the current KA crew), you all have my best wishes for a great start to the school year.
For regular updates, follow me on Twitter at @bjfr and for my papers, presentations and so forth, visit EdTechResearcher.