Federal

Scholars Test Emotion-Sensitive Tutoring Software

‘Intelligent’ Systems Respond to Students’ Cues
By Debra Viadero — December 22, 2009 6 min read
BRIC ARCHIVE
  • Save to favorites
  • Print

Includes updates and/or revisions.

Still, no one expects the technology to ever replace the real thing.

Then she responds encouragingly, “Let’s read again what the problem is asking.”

This scene would not be out of place in most educational environments were it not for one important detail: Jane is a character on a computer screen.

Developed by teams of researchers at the University of Massachusetts Amherst and Arizona State University in Tempe, the computer tutor in this scenario is part of a growing number of research projects around the country looking to build a social and emotional support system into intelligent-tutoring systems. Powered by artificial-intelligence technology, intelligent tutors have been around for decades, and some are beginning to make their way into regular classroom use. Cognitive Tutor, one of the oldest and best known of such programs, for instance, now operates in 2,600 schools around the country.

But programming the systems to detect and respond to students’ emotions in the same way that human tutors do is a new wrinkle for the field and one that developers hope will enhance the technology’s educational potential.

More Resources

“Human tutors are believed to be the highest form of instruction,” said James C. Lester, a computer scientist who is exploring such ideas at North Carolina State University in Raleigh. “Why? The hypothesis is that there is very strong affective feedback going on. Five years ago, this was something the field would’ve thought was far out, but now it’s getting a lot more resources thrown at it.”

Major funders of research into imbuing digital tutors with the ability to detect and respond to students’ emotions include the National Science Foundation and the U.S. Department of Education’s Institute of Education Sciences.

The jury is still out on whether students learn more with emotion-enhanced computer tutors than they do with the other kinds of computer tutors, but some early results from small experiments around the country show promise.

In the project developed by the University of Massachusetts and Arizona State, which has been tested so far in a high school and in two schools of education, students’ passing rates on state tests were 10 percent higher after a week of lessons with the tutor than they were for peers who spent the same amount of time learning geometry in a regular classroom, according to Beverly Park Woolf, a research professor in computer science at the University of Massachusetts Amherst and a leading researcher on the project.

Her collaborators include Ivon M. Arroyo, also a research scientist at the University of Massachusetts, and Winslow Burleson, an assistant professor in the arts, media, and engineering program at Arizona State.

Their project is built around an intelligent-tutoring system known as Wayang Outpost, which uses real-world problem-solving tasks to teach geometry. The system picks up on students’ emotional states through hundreds of sensors embedded in the computer, students’ chairs, and other aspects of the students’ learning environment.

Sensors worn like a bracelet around the wrist detect changes in students’ pulse and in moisture levels on the surface of the skin. Sensors embedded in the chair cushions identify nine different postures a learner might take. Leaning forward, for example, suggests engagement, while a student leaning to the side might be bored or frustrated. On the computer mouse, pressure-sensitive sensors signal whether a student is squeezing harder in possible frustration.

The researchers also collect data on students’ emotions through a video camera embedded on the computer. It trains its focus on the student’s eyebrows, mouth, and nose, discerning whether the learner is smiling, frowning, or yawning.

In combination, the researchers find, the data collected through the sensors enables the system to correctly identify variation in students’ emotions more than half the time.

The program responds to students’ emotions in the form of “pedagogical agents”—which are animated characters, such as Jane—who mirror those emotions and offer an appropriate response. Building on the work of Carol S. Dweck, the Stanford University psychologist who argues that children work harder when they believe that intelligence is malleable rather than a fixed, inborn trait, the tutors also give feedback that emphasizes the value of effort.

“Hey, congratulations!,” Jane or “Jake” or “Isabel” might say, “Your effort paid off. You got it right!”

What the researchers have found in some of the studies so far is that the emotion-sensitive tutors, besides boosting students’ average achievement, seem to lead to improvements in the way that students think about math and their own math abilities. They are more likely after their tutoring sessions to agree, for instance, with statements saying “mathematics is an important topic” and to believe they are good at it. Students also seem to get bored with the tutor less quickly than they do with the unenhanced system.

Confidence Builder

Not all students liked the tutor, though. Students can switch off the animated tutors and boys, impatient to move along, often did. But girls, who started out in the studies expressing lower levels of confidence in their abilities than the boys did, also ended up feeling more confident than the boys when they used the tutor. The system also seems to be particularly effective with students who are classified to be in need of special education, Ms. Woolf said.

Ms. Woolf and her research partners also have found that boys and girls respond differently to pedagogical agents of a different gender. Researchers are now exploring what happens when the computer characters reflect the cultural orientations of their own racial and ethnic group.

“American Indians, for example, have a culture that talks about collaboration and not sticking out, which is opposite from the way most classrooms work, so the character can talk to American Indian students more about collaboration in the classroom,” said Ms. Woolf.

“In 20 years,” she said, “I think we will have personalized tutoring in the classroom.”

But other researchers, such as Arthur C. Graesser and Sidney D’Mello of the University of Memphis, have not found that students respond much differently to characters of different gender or racial or ethnic groups.

“We think it’s more what is said at the right time than what the agent looks like,” said Mr. Graesser, a psychology professor, “but that may not be true if students have to stay with a certain agent for weeks or months.”

In his work, Mr. Graesser has also found that the emotionally enhanced systems work better for poorer learners than they do for high achievers—but only after about half an hour.

‘Shake Up’ Comments

“You don’t want a tutor to get emotional too quick,” he said. “Students seem to need conventional action to build up some bonding.”

Mr. Graesser and Mr. D’Mello experimented with injecting a dose of personality into their tutors.

“We called it a ‘shake up’ tutor,” he said. “If a student is frustrated, the tutor says, ‘I can see you’re frustrated. I thought you could handle this but I guess I need to rethink that.’ ”

But he said that approach hasn’t yet “panned out” in experimentation.

“We thought it might be more engaging for a certain class of students—maybe college students—who are a little more sophisticated.”

While the systems can be cumbersome for students to use and expensive to develop at first, all the experts said, they are becoming less so all the time.

“The sensors were very cumbersome, but now they’re just kind of cumbersome,” said Mr. Lester of North Carolina State, where researchers are integrating emotional feedback into game-based learning environments.

Within five to 10 years, he and others predict, costs may be low enough to make widespread use practical. Still, no one expects the technology to ever replace the real thing.

“There’s no way that I envision a class of 30 children going through the entire day on little machines,” Ms. Woolf said. “This will augment teaching. But as these systems become trustworthy, I absolutely think there will be a place in the classroom for them.”

A version of this article appeared in the January 06, 2010 edition of Education Week as Projects Build Social, Emotional Support Into Computer Tutoring

Events

School Climate & Safety K-12 Essentials Forum Strengthen Students’ Connections to School
Join this free event to learn how schools are creating the space for students to form strong bonds with each other and trusted adults.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
Future-Proofing Your School's Tech Ecosystem: Strategies for Asset Tracking, Sustainability, and Budget Optimization
Gain actionable insights into effective asset management, budget optimization, and sustainable IT practices.
Content provided by Follett Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Budget & Finance Webinar
Innovative Funding Models: A Deep Dive into Public-Private Partnerships
Discover how innovative funding models drive educational projects forward. Join us for insights into effective PPP implementation.
Content provided by Follett Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal K-12 Leaders Denounce Antisemitism But Reject That It's Rampant in Schools
Three school district leaders said they're committed to rooting out antisemitism during a hearing in Congress.
6 min read
From left, David Banks, chancellor of New York Public schools, speaks next to Karla Silvestre, President of the Montgomery Count (Md.) Board of Education, Emerson Sykes, Staff Attorney with the ACLU, and Enikia Ford Morthel, Superintendent of the Berkeley United School District, during a hearing on antisemitism in K-12 public schools, at the House Subcommittee on Early Childhood, Elementary, and Secondary Education, on May 8, 2024, on Capitol Hill in Washington.
From left, David Banks, chancellor of New York City schools, speaks next to Karla Silvestre, president of the Montgomery County, Md., school board; Emerson Sykes, staff attorney with the ACLU; and Enikia Ford Morthel, superintendent of the Berkeley Unified school district in Berkeley, Calif., during a hearing on antisemitism in K-12 public schools, at the House Subcommittee on Early Childhood, Elementary, and Secondary Education, on May 8, 2024, in Washington.
Jacquelyn Martin/AP
Federal Miguel Cardona in the Hot Seat: 4 Takeaways From a Contentious House Hearing
FAFSA, rising antisemitism, and Title IX dominated questioning at a U.S. House hearing with Education Secretary Miguel Cardona.
6 min read
Secretary of Education Miguel Cardona testifies during a House Committee on Education and Workforce hearing on Capitol Hill, Tuesday, May 7, 2024, in Washington.
U.S. Secretary of Education Miguel Cardona testifies during a House Committee on Education and Workforce hearing on Capitol Hill on May 7 in Washington.
Mariam Zuhaib/AP
Federal Arming Teachers Could Cause 'Accidents and More Tragedy,' Miguel Cardona Says
"This is not in my opinion a smart option,” the education secretary said at an EdWeek event.
4 min read
U.S. Secretary of Education Miguel Cardona speaks during Education Week’s 2024 Leadership Symposium at the Hyatt Regency Crystal City in Arlington, Va., on May 2, 2024.
U.S. Secretary of Education Miguel Cardona speaks during Education Week’s 2024 Leadership Symposium at the Hyatt Regency Crystal City in Arlington, Va., on May 2, 2024.
Sam Mallon/Education Week
Federal Opinion Should Migrant Families Pay Tuition for Public School?
The answer must reflect an outlook that is pro-immigration, pro-compassion, and pro-law and order, writes Michael J. Petrilli.
Michael J. Petrilli
4 min read
Image of a pencil holder filled with a variety of colored pencils that match the background with international flags.
Laura Baker/Education Week via Canva