In my HarvardX datasets, and in the dataset of most xMOOCs, there is a column called something like “got_certificate.” It’s a binary field in the dataset, and participants have a 0 if they have not earned a certificate and a 1 if they do.
In analyzing MOOCs, one of people’s favorite things to do is count the number of 1s in the got_cert column, as a representation of how many people “completed the course.” Another favorite thing for MOOC pundits to do is divide the sum of the 1s in the got_cert column by the total number of registrants, to get the completion rate, and of course 100 minus the completion rate is the attrition rate. Quite a bit of the conversation about MOOCs is driven by these numbers.
Let me share a story about one of my favorite 0s.
There is a delightful young woman--I’ll call her Jane--in my undergraduate course on education at MIT, and in a recent class session I was sitting in the back doing some small group work with her. My co-instructor Wendy Huang was facilitating a fabulous lesson contrasting instructionist and constructivist approaches to mathematics instruction. Jane and I were in a group were working through some case studies of different teachers applying diverse approaches. Jane brought up that she had taken Jo Boaler’s How to Learn Math course from Stanford this summer, but she had dropped out about 2/3 of the way through as she got too busy with getting ready to head back to campus and other things. So Jane’s a 0.
She then started talking about the case studies of teaching we were reading, and she started bringing up some of the reading and learning she had done about Carol Dweck’s work on mindsets. She started applying some specific ideas from the How to Learn Math Course--thoughtfully, appropriately, and without prompting--to our discussion of these teaching case studies. I then asked her what else she had learned, and she started talking about math talk, strategies for engaging students, and some of the other strategies she had learned months ago. She was hoping to get back around to finishing the rest of the pieces of the course when she had some time.
So, months after taking a short online class, Jane was capable of recalling relevant materials and transferring concepts in a useful way in a social context to address a parallel but not identical problem. That’s meaningful learning in my book. When we use measures of completion rates as a crude summary of learning in open online courses, we implictly assert that because Jane didn’t “complete the course” we should infer or assume that her learning experience is inferior to those who did.
In most of the HarvardX courses that we’ve started to examine, there are accounts whose activity entirely consists of answering problems and assessments without doing any of the learning work. Some of these 1s in the got_cert column might be people who already know the material but are quizzing themselves, some might be bots, some might be participating in behavior like taking tests over and over for practice, behavior that would be considered cheating in certain kinds of contexts. Just as there are Jane’s amongst the 0s, there are bots amongst the 1s.
Some of my colleagues at MIT’s Teaching and Learning Lab and at HarvardX have written a thoughtful paper, titled “Changing Course,” arguing that many of the words that we use to describe MOOCs are borrowed from analogous residential college courses, and we transfer the meaning of those words across settings inappropriately. A course is not a course. Enrolling is not enrolling. Completing isn’t completing. Dropping out isn’t dropping out. The behaviors sometimes look similar, and we can’t invent a whole new set of words to describe things (“Jane aardvarked about 2/3 of the way through the course”), but we have to be keenly aware that the words we are borrowing from residential education have different connotations and significance in the different learning environments that we are creating.
If you are following the discourse of MOOCs, when you hear about those MOOC dropouts, think about Jane. Crude measures of attrition are masking potentially important dimensions of learning. If you are a MOOC researcher, one of the most promising avenues of inquiry is figuring out how many Janes there are in our pile of 0s, and how much they are learning and taking away from these courses.
For regular updates, follow me on Twitter at @bjfr and for my publications, C.V., and online portfolio, visit EdTechResearcher.