Student Achievement

New Breed of Digital Tutors Yielding Learning Gains

By Debra Viadero — April 02, 2007 7 min read

Struggling algebra students in the Everett, Wash., school district get help from special tutors who diagnose their weaknesses, tailor instruction to their needs, and provide on-the-spot feedback—all with an inhuman degree of patience.

That’s inhuman literally: The tutors are computers. Three years ago, the district started employing Cognitive Tutor, a series of computer programs based on artificial intelligence that were developed by researchers from Carnegie-Mellon University in Pittsburgh. The programs provide an alternative form of math instruction to secondary school students who haven’t succeeded in regular classrooms.

The experience proved so successful that officials in the 20,000-student district have expanded the program. Now, all of Everett’s high school students have a choice in signing up for Algebra 1, Algebra 2, and geometry: a traditional class or one that mixes teacher-led lessons with a sizable dose of machine-based tutoring.

“I was a math teacher for 20 years,” said Mary Ann Stine, the district’s director of curriculum and instruction, “and the question I have to ask is: How come this is so much more successful than I was?”

Used in 1,500 districts nationwide, the Cognitive Tutor programs are the best-known of a small but expanding contingent of “intelligent tutors” that are making their way from the laboratory into college and K-12 classrooms.

Such artificial-intelligence-based programs now are on the market or in development for teaching chemistry, physics, foreign languages, reading, and computer science, among other subjects, and for grading essays.

“What distinguishes intelligent tutors from integrated learning systems or skill-building software is that the tutors sort of both scaffold and support more complex cognitive processes,” said Margaret Honey, the director of the New York City-based Center for Children and Technology. “Well-designed tutors are smart enough to know there’s not a single way to solve a problem, and that’s what makes them ‘intelligent.’ ”

No Rivals for Live Tutors

Since the 1970s, the National Science Foundation, the Pentagon, and the U.S. Department of Education have opened their wallets to seed research and development of intelligent-tutoring systems.

The agencies have put up the money, in part, because studies show that the systems are promising.

The Cognitive Tutor Algebra program, for instance, is one of just two middle school mathematics programs to get a “positive” effectiveness rating in 2004 from the What Works Clearinghouse, a federally funded effort to vet research on educational programs and practices.

Studies suggest that, on average, students who use Cognitive Tutor make learning gains that roughly translate into the equivalent of as much as one letter grade—the difference, in other words, between an A and a B.

The improvements seem greatest, the research also suggests, in the area of complex problem-solving and for students who start out with weaker math or English-language skills. Similar gains are being reported for other kinds of intelligent-tutoring systems as well.

Those kinds of improvements still don’t rival one-on-one lessons from a flesh-and-bones teacher, though. According to an influential 1984 study review, performance improves by the equivalent of about two letter grades when students get individual lessons from human tutors.

But the level of progress also depends on how skilled the tutor is, noted Kenneth R. Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon and one of the developers of Cognitive Tutor.

“Our goal isn’t to replace teaching,” said Mr. Koedinger, who also co-directs the Pittsburgh Science of Learning Center, a $25 million operation run jointly by the University of Pittsburgh and CMU that uses intelligent-tutoring systems to study learning. “It’s to give teachers more time to do what they do best.”

Most intelligent-tutoring systems are built as supplements to classroom lessons. Even in Cognitive Tutor’s full-course programs in algebra and geometry, students spend 40 percent of their time on computers and 60 percent in classrooms, where teachers use lessons developed by the Pittsburgh researchers.

“The better contrast to use might be a textbook,” said Mr. Koedinger. “With textbooks, students don’t get feedback on solutions.”

Projects in Pipeline

Cognitive Tutor was born out of work begun by John R. Anderson, a prominent cognitive scientist at Carnegie Mellon, in the 1970s. Mr. Anderson wanted to use the system to test a framework for modeling how advanced students process and learn information.

In 1998, though, Mr. Anderson, Mr. Koedinger, and their research partners had a string of products that were ready for mass distribution. They formed Carnegie Learning Inc., a Pittsburgh-based company, to promote Cognitive Tutor nationwide.

Meanwhile, other researchers were also experimenting with the technology and adding twists. Some of the other innovations in the production pipeline include:

Learn more about Project LISTEN and AutoTutor.

• Project LISTEN, a system developed by Carnegie Mellon researcher Jack Mostow that employs speech-synthesizing technology to “listen” to students read and give them feedback. The computer also assigns students new stories based on their progress.

Studies so far show that elementary school students who use the program in addition to their regular reading instruction for as little as 20 minutes a day show learning improvements equivalent to about half to two-thirds of a letter grade better than their regular-classroom counterparts.

• Online intelligent-tutoring systems in chemistry and applied mathematics that allow students to plug in their own homework problems, rather than rely on computer-generated tasks. Developed by Quantum Simulations Inc., of Murrysville, Pa., the chemistry programs, which are already on the market, have been tested with high school and college students.

The studies show that high school students who used the tutor at home were better at balancing chemical equations than were their peers who did not use the technology.

The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.
—Image courtesy of Institute for Intelligent Systems

Talking Head: The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.

• AutoTutor, a system developed by researchers at the University of Memphis, in Tennessee, and Carnegie Mellon that features an on-screen talking head that conducts a dialogue with students. Developed primarily for physics and computer-literacy studies at the university level, the program is now being tested with students in grades 8-11 as well.

“The goal is to help students reason,” said Arthur C. Graesser, the Memphis researcher heading that project. “When we compare it to just reading texts, we find that the AutoTutor helps by about one letter grade.”

Reading Body Language

An open question is whether the systems work as well in subjects whose subject matter is less specified, or more the focus of contention, than in math and science.

Experts also wonder how widespread such programs can become, given the time and expense they take to develop. Cognitive Tutor, for instance, was developed and refined over more than 20 years. Even now, researchers continue to tweak it.

“For every hour of instruction, there’s hundreds of hours spent in development,” said Mr. Anderson of Carnegie Mellon.

Yet Carnegie Learning has reached a point now where it can market its product for $30 to $60 per student and earn enough to keep devising new products, according to Steven B. Ritter, another of the company’s founders.

Researchers are also discovering, though, that intelligent-tutoring systems are as valuable as research tools as they are as teaching instruments. That’s because the systems can track student behaviors with a precision not available to classroom researchers, who rely on twice-a-year surveys or occasional classroom observations for their data.

With AutoTutor, Mr. Graesser says, researchers have embedded sensing devices to track students’ facial expressions, voice signals, and posture.

“We’re getting good at detecting boredom versus frustration versus confusion,” he said. “The more difficult the AutoTutor is, the better we get students to learn, and the less they like it.” That may not be so different, after all, from their experiences with human teachers.

Related Tags:

Coverage of education research is supported in part by a grant from the Spencer Foundation
A version of this article appeared in the April 04, 2007 edition of Education Week as New Breed of Digital Tutors Yielding Learning Gains


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Webinar
Interactive Learning Best Practices: Creative Ways Interactive Displays Engage Students
Students and teachers alike struggle in our newly hybrid world where learning takes place partly on-site and partly online. Focus, engagement, and motivation have become big concerns in this transition. In this webinar, we will
Content provided by Samsung
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Educator-Driven EdTech Design: Help Shape the Future of Classroom Technology
Join us for a collaborative workshop where you will get a live demo of GoGuardian Teacher, including seamless new integrations with Google Classroom, and participate in an interactive design exercise building a feature based on
Content provided by GoGuardian
School & District Management Live Online Discussion A Seat at the Table With Education Week: What Did We Learn About Schooling Models This Year?
After a year of living with the pandemic, what schooling models might we turn to as we look ahead to improve the student learning experience? Could year-round schooling be one of them? What about online

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Achievement Opinion Want to Improve Learning Outcomes? Give Students More Time
An extended school year, multiage classrooms, and other strategies can address learning loss, says Penn Graduate School of Education dean.
Pam Grossman
5 min read
Illustration of students working on laptops.
Laura Lannes for Education Week
Student Achievement 3 Studies for District Leaders to Read as They Plan for Learning Recovery
How can districts plan for programs that help catch up students affected academically by the pandemic? These three studies provide clues.
4 min read
26extendresearch 1209998304 blue
Student Achievement Where Can We Find Lots of Tutors? Bill in Congress Would Deploy Teachers-in-Training
A bipartisan proposal in the Senate eyes teacher-candidates as a promising source to help districts scale up intensive tutoring programs.
4 min read
Tutoring cost rising
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Achievement Whitepaper
Expert Research: How Severe Is Unfinished Learning?
This recently released report offers the first large-scale look at how the pandemic has impacted learning across all students, widened ex...
Content provided by Curriculum Associates