Education Opinion

MOOCs and the Science of Learning

By Justin Reich — July 08, 2014 2 min read
  • Save to favorites
  • Print

I gave a talk last week at the Berkman Center, where I tried to summarize the state of MOOC research as it relates specifically to learning.

The short version is this: we have terabytes of data about what people clicked, and very little understanding about what changed in people’s heads. It’s hard to look at the science of learning in MOOCs, because we don’t know much about what they are learning. The visual short version is this vizthink from the great Willow Bl00.

The lowest hanging research fruit tends to be more user experience research than learning research. There are lots of studies showing that people who do stuff in one part of a MOOC do more stuff in another part of the course, but very few studies that can characterize what kinds of competencies people came into a course with and how those competencies changed over time.

The main problem is that designing good assessments is really hard and time-consuming anywhere in education, and particular difficult when you are trying to assess thousands of people instantly and without human experts. At the same time, hoovering up clickstream data and running it through clustering algortihms or regression models is relatively easy. So if you are young scholar in a new field trying to make a name for yourself quickly (myself included), you grab the low-hanging fruit.

So I have three suggestions for reshaping the field. First, we need to help course developers create better measures of people’s competencies, so we have outcome variables that are better than engagement metrics. Then, since learning is a measure of the change between two levels of competency, we need to at least be measuring competency at the beginning and end of courses, not just at the end. This means finding more ways to evalute students throughout our courses. Finally, we need to do research that builds causal chains of reasoning to connect the moves we make inside courses with the learning measures that we can obtain. On the quantiative side, that meaning introducing experimental variation into courses so that we can better understand what kinds of instructional moves lead to what kind of learning. On the qualitative side, that means doing more interviews and observations with MOOC learners, so we have a richer set of hypotheses about the mechanisms behind open online learning, especially the learning happening in the untrackable space beyond our platforms.

It has been totally fine and natural that in the early years of MOOC research, as we try to make sense of the data and the courses, people are aiming for the low hanging fruit. But the critque that hangs over that work is that we are unearthing the obvious. The actionable advice we can offer to course developers thus far is “get your students to do stuff.” As we head into a new year, it’s a great opportunity to imagine what kinds of research would let us make claims not just about what students are doing, but about what they are learning.

The full video is below.

Many thanks to those who asked questions, shared their thoughts, tweeted and said hello afterwards.

For regular updates, follow me on Twitter at @bjfr and for my papers, presentations and so forth, visit EdTechResearcher.

The opinions expressed in EdTech Researcher are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Attendance Awareness Month: The Research Behind Effective Interventions
More than a year has passed since American schools were abruptly closed to halt the spread of COVID-19. Many children have been out of regular school for most, or even all, of that time. Some
Content provided by AllHere

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Schools Get the Brunt of Latest COVID Wave in South Carolina
In the past few weeks, South Carolina has set records for COVID-19 hospitalizations and new cases have approached peak levels of last winter.
4 min read
Two Camden Elementary School students in masks listen as South Carolina Gov. Henry McMaster talks about steps the school is taking to fight COVID-19, Wednesday, Sept. 15, 2021, in Camden, S.C. McMaster has adamantly and repeatedly come out against requiring masks in schools even as the average number of daily COVID-19 cases in the state has risen since early June. (AP Photo/Jeffrey Collins)
Education More States Are Requiring Schools to Teach Native American History and Culture
Advocates say their efforts have gained some momentum with the nation’s reckoning over racial injustice since the killing of George Floyd.
3 min read
A dancer participates in an intertribal dance at Schemitzun on the Mashantucket Pequot Reservation in Mashantucket, Conn., Saturday, Aug. 28, 2021. Connecticut and a handful of other states have recently decided to mandate students be taught about Native American culture and history. (AP Photo/Jessica Hill)
Education Judge's Temporary Order Allows Iowa Schools to Mandate Masks
A federal judge ordered the state to immediately halt enforcement of a law that prevents school boards from ordering masks to be worn.
4 min read
Iowa Gov. Kim Reynolds speaks to reporters following a news conference, Thursday, Aug. 19, 2021, in West Des Moines, Iowa. Reynolds lashed out at President Joe Biden Thursday after he ordered his education secretary to explore possible legal action against states that have blocked school mask mandates and other public health measures meant to protect students against COVID-19. Reynolds, a Republican, has signed a bill into law that prohibits school officials from requiring masks, raising concerns as delta variant virus cases climb across the state and schools resume classes soon. (AP Photo/Charlie Neibergall)
Education Hurricane Ida Deals New Blow to Louisiana Schools Struggling to Reopen
The opening of the school year offered teachers a chance to fully assess the pandemic's effects, only to have students forced out again.
8 min read
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021. Louisiana students, who were back in class after a year and a half of COVID-19 disruptions kept many of them at home, are now missing school again after Hurricane Ida. A quarter-million public school students statewide have no school to report to, though top educators are promising a return is, at most, weeks away, not months.
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021.
John Locher/AP