Even for a teacher with eyes in the back of her head, it is not always possible to see who is on task and who is staring out the window at the sky.
So in recent years, researchers have been experimenting with cameras and computers to identify facial expressions and body language associated with lower and higher levels of student engagement.
For instance, in an article published in April in the peer-reviewed journal IEEE Transactions on Affective Computing, researchers had humans observe video clips of college students learning to play mentally challenging games on an iPad. The researchers then “taught” computers to use the clips to make judgments about students’ levels of engagement. The computer classifications were just as reliable as those of human observers. Students also took tests before and after the computer games on the skills the games were intended to teach. And researchers found that the video-based engagement scores predicted the post-test scores better than the pretest scores could. The work is a project of the National Science Foundation-funded Temporal Dynamics of Learning Center at the University of California, San Diego.
In actual K-12 settings, camera-based engagement measures are still rare to nonexistent. But UC-San Diego research professor Javier Movellan, who helped conduct the iPad study, believes the technology is ready for prime time in schools. His evidence is a focus group conducted by Interscope Research and the for-profit company Emotient, which Mr. Movellan co-founded to develop and distribute software that recognizes human expressions.
The focus-group setting was physically similar to that of many classrooms in that 35 people sat in rows facing the front of the room, rather than in front of a computer. Yet researchers successfully used cameras and software to classify the facial expressions of 35 Denver Broncos and Seattle Seahawks fans as they watched the 2014 Super Bowl and the advertisements that ran during the game. Based on the facial images, the software rated the emotional impact of each commercial and the growing frustration of the Denver fans as their team fell behind.
Mr. Movellan suggested the expression-recognition software could have multiple potential uses in the K-12 field—for teachers, educational researchers, and computerized instruction.
Of course, anytime cameras appear in classrooms, privacy issues emerge. At this point, Mr. Movellan said, the technology cannot tell one person from another. But he said privacy was “a very important concern” and one that “students and teachers and educators eventually need to figure out how to deal with.”
