Student Achievement

New Breed of Digital Tutors Yielding Learning Gains

By Debra Viadero — April 02, 2007 7 min read
  • Save to favorites
  • Print

Struggling algebra students in the Everett, Wash., school district get help from special tutors who diagnose their weaknesses, tailor instruction to their needs, and provide on-the-spot feedback—all with an inhuman degree of patience.

That’s inhuman literally: The tutors are computers. Three years ago, the district started employing Cognitive Tutor, a series of computer programs based on artificial intelligence that were developed by researchers from Carnegie-Mellon University in Pittsburgh. The programs provide an alternative form of math instruction to secondary school students who haven’t succeeded in regular classrooms.

The experience proved so successful that officials in the 20,000-student district have expanded the program. Now, all of Everett’s high school students have a choice in signing up for Algebra 1, Algebra 2, and geometry: a traditional class or one that mixes teacher-led lessons with a sizable dose of machine-based tutoring.

“I was a math teacher for 20 years,” said Mary Ann Stine, the district’s director of curriculum and instruction, “and the question I have to ask is: How come this is so much more successful than I was?”

Used in 1,500 districts nationwide, the Cognitive Tutor programs are the best-known of a small but expanding contingent of “intelligent tutors” that are making their way from the laboratory into college and K-12 classrooms.

Such artificial-intelligence-based programs now are on the market or in development for teaching chemistry, physics, foreign languages, reading, and computer science, among other subjects, and for grading essays.

“What distinguishes intelligent tutors from integrated learning systems or skill-building software is that the tutors sort of both scaffold and support more complex cognitive processes,” said Margaret Honey, the director of the New York City-based Center for Children and Technology. “Well-designed tutors are smart enough to know there’s not a single way to solve a problem, and that’s what makes them ‘intelligent.’ ”

No Rivals for Live Tutors

Since the 1970s, the National Science Foundation, the Pentagon, and the U.S. Department of Education have opened their wallets to seed research and development of intelligent-tutoring systems.

The agencies have put up the money, in part, because studies show that the systems are promising.

The Cognitive Tutor Algebra program, for instance, is one of just two middle school mathematics programs to get a “positive” effectiveness rating in 2004 from the What Works Clearinghouse, a federally funded effort to vet research on educational programs and practices.

Studies suggest that, on average, students who use Cognitive Tutor make learning gains that roughly translate into the equivalent of as much as one letter grade—the difference, in other words, between an A and a B.

The improvements seem greatest, the research also suggests, in the area of complex problem-solving and for students who start out with weaker math or English-language skills. Similar gains are being reported for other kinds of intelligent-tutoring systems as well.

Those kinds of improvements still don’t rival one-on-one lessons from a flesh-and-bones teacher, though. According to an influential 1984 study review, performance improves by the equivalent of about two letter grades when students get individual lessons from human tutors.

But the level of progress also depends on how skilled the tutor is, noted Kenneth R. Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon and one of the developers of Cognitive Tutor.

“Our goal isn’t to replace teaching,” said Mr. Koedinger, who also co-directs the Pittsburgh Science of Learning Center, a $25 million operation run jointly by the University of Pittsburgh and CMU that uses intelligent-tutoring systems to study learning. “It’s to give teachers more time to do what they do best.”

Most intelligent-tutoring systems are built as supplements to classroom lessons. Even in Cognitive Tutor’s full-course programs in algebra and geometry, students spend 40 percent of their time on computers and 60 percent in classrooms, where teachers use lessons developed by the Pittsburgh researchers.

“The better contrast to use might be a textbook,” said Mr. Koedinger. “With textbooks, students don’t get feedback on solutions.”

Projects in Pipeline

Cognitive Tutor was born out of work begun by John R. Anderson, a prominent cognitive scientist at Carnegie Mellon, in the 1970s. Mr. Anderson wanted to use the system to test a framework for modeling how advanced students process and learn information.

In 1998, though, Mr. Anderson, Mr. Koedinger, and their research partners had a string of products that were ready for mass distribution. They formed Carnegie Learning Inc., a Pittsburgh-based company, to promote Cognitive Tutor nationwide.

Meanwhile, other researchers were also experimenting with the technology and adding twists. Some of the other innovations in the production pipeline include:

Learn more about Project LISTEN and AutoTutor.

• Project LISTEN, a system developed by Carnegie Mellon researcher Jack Mostow that employs speech-synthesizing technology to “listen” to students read and give them feedback. The computer also assigns students new stories based on their progress.

Studies so far show that elementary school students who use the program in addition to their regular reading instruction for as little as 20 minutes a day show learning improvements equivalent to about half to two-thirds of a letter grade better than their regular-classroom counterparts.

• Online intelligent-tutoring systems in chemistry and applied mathematics that allow students to plug in their own homework problems, rather than rely on computer-generated tasks. Developed by Quantum Simulations Inc., of Murrysville, Pa., the chemistry programs, which are already on the market, have been tested with high school and college students.

The studies show that high school students who used the tutor at home were better at balancing chemical equations than were their peers who did not use the technology.

The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.
—Image courtesy of Institute for Intelligent Systems

Talking Head: The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.

• AutoTutor, a system developed by researchers at the University of Memphis, in Tennessee, and Carnegie Mellon that features an on-screen talking head that conducts a dialogue with students. Developed primarily for physics and computer-literacy studies at the university level, the program is now being tested with students in grades 8-11 as well.

“The goal is to help students reason,” said Arthur C. Graesser, the Memphis researcher heading that project. “When we compare it to just reading texts, we find that the AutoTutor helps by about one letter grade.”

Reading Body Language

An open question is whether the systems work as well in subjects whose subject matter is less specified, or more the focus of contention, than in math and science.

Experts also wonder how widespread such programs can become, given the time and expense they take to develop. Cognitive Tutor, for instance, was developed and refined over more than 20 years. Even now, researchers continue to tweak it.

“For every hour of instruction, there’s hundreds of hours spent in development,” said Mr. Anderson of Carnegie Mellon.

Yet Carnegie Learning has reached a point now where it can market its product for $30 to $60 per student and earn enough to keep devising new products, according to Steven B. Ritter, another of the company’s founders.

Researchers are also discovering, though, that intelligent-tutoring systems are as valuable as research tools as they are as teaching instruments. That’s because the systems can track student behaviors with a precision not available to classroom researchers, who rely on twice-a-year surveys or occasional classroom observations for their data.

With AutoTutor, Mr. Graesser says, researchers have embedded sensing devices to track students’ facial expressions, voice signals, and posture.

“We’re getting good at detecting boredom versus frustration versus confusion,” he said. “The more difficult the AutoTutor is, the better we get students to learn, and the less they like it.” That may not be so different, after all, from their experiences with human teachers.

Related Tags:

Coverage of education research is supported in part by a grant from the Spencer Foundation
A version of this article appeared in the April 04, 2007 edition of Education Week as New Breed of Digital Tutors Yielding Learning Gains


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
What’s Next for Teaching and Learning? Key Trends for the New School Year
The past 18 months changed the face of education forever, leaving teachers, students, and families to adapt to unprecedented challenges in teaching and learning. As we enter the third school year affected by the pandemic—and
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Achievement What the Research Says Here's One Way to Improve Students' Reading Scores: Get Them Eyeglasses
Schools need to do more than just vision screenings, new research says, especially after students have spent so much time learning on screen.
5 min read
Image of eyeglasses and an eye examine chart with letters.
gchutka/E+
Student Achievement Schools Straddle the Pandemic and Familiar Headwinds in Quest to Boost Quality
The latest Quality Counts summative grades show stubbornly average performance by the nation's schools overall, despite pockets of promise.
1 min read
Illustration of C letter grade
Getty
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Whitepaper
KEEP ELEMENTARY STUDENTS ON COURSE
Gale partnered with Project Tomorrow® to carry out a nationwide school study that examined the efficacy of using cross-curricular educati...
Content provided by Gale
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Achievement Whitepaper
The Key to Addressing Pandemic Learning Loss
The pandemic was a setback for many students. How do we help educators get students back on track? Read Lexia’s blog, Equipping Educators...
Content provided by Lexia Learning