Student Achievement

New Breed of Digital Tutors Yielding Learning Gains

By Debra Viadero — April 02, 2007 7 min read
  • Save to favorites
  • Print

Struggling algebra students in the Everett, Wash., school district get help from special tutors who diagnose their weaknesses, tailor instruction to their needs, and provide on-the-spot feedback—all with an inhuman degree of patience.

That’s inhuman literally: The tutors are computers. Three years ago, the district started employing Cognitive Tutor, a series of computer programs based on artificial intelligence that were developed by researchers from Carnegie-Mellon University in Pittsburgh. The programs provide an alternative form of math instruction to secondary school students who haven’t succeeded in regular classrooms.

The experience proved so successful that officials in the 20,000-student district have expanded the program. Now, all of Everett’s high school students have a choice in signing up for Algebra 1, Algebra 2, and geometry: a traditional class or one that mixes teacher-led lessons with a sizable dose of machine-based tutoring.

“I was a math teacher for 20 years,” said Mary Ann Stine, the district’s director of curriculum and instruction, “and the question I have to ask is: How come this is so much more successful than I was?”

Used in 1,500 districts nationwide, the Cognitive Tutor programs are the best-known of a small but expanding contingent of “intelligent tutors” that are making their way from the laboratory into college and K-12 classrooms.

Such artificial-intelligence-based programs now are on the market or in development for teaching chemistry, physics, foreign languages, reading, and computer science, among other subjects, and for grading essays.

“What distinguishes intelligent tutors from integrated learning systems or skill-building software is that the tutors sort of both scaffold and support more complex cognitive processes,” said Margaret Honey, the director of the New York City-based Center for Children and Technology. “Well-designed tutors are smart enough to know there’s not a single way to solve a problem, and that’s what makes them ‘intelligent.’ ”

No Rivals for Live Tutors

Since the 1970s, the National Science Foundation, the Pentagon, and the U.S. Department of Education have opened their wallets to seed research and development of intelligent-tutoring systems.

The agencies have put up the money, in part, because studies show that the systems are promising.

The Cognitive Tutor Algebra program, for instance, is one of just two middle school mathematics programs to get a “positive” effectiveness rating in 2004 from the What Works Clearinghouse, a federally funded effort to vet research on educational programs and practices.

Studies suggest that, on average, students who use Cognitive Tutor make learning gains that roughly translate into the equivalent of as much as one letter grade—the difference, in other words, between an A and a B.

The improvements seem greatest, the research also suggests, in the area of complex problem-solving and for students who start out with weaker math or English-language skills. Similar gains are being reported for other kinds of intelligent-tutoring systems as well.

Those kinds of improvements still don’t rival one-on-one lessons from a flesh-and-bones teacher, though. According to an influential 1984 study review, performance improves by the equivalent of about two letter grades when students get individual lessons from human tutors.

But the level of progress also depends on how skilled the tutor is, noted Kenneth R. Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon and one of the developers of Cognitive Tutor.

“Our goal isn’t to replace teaching,” said Mr. Koedinger, who also co-directs the Pittsburgh Science of Learning Center, a $25 million operation run jointly by the University of Pittsburgh and CMU that uses intelligent-tutoring systems to study learning. “It’s to give teachers more time to do what they do best.”

Most intelligent-tutoring systems are built as supplements to classroom lessons. Even in Cognitive Tutor’s full-course programs in algebra and geometry, students spend 40 percent of their time on computers and 60 percent in classrooms, where teachers use lessons developed by the Pittsburgh researchers.

“The better contrast to use might be a textbook,” said Mr. Koedinger. “With textbooks, students don’t get feedback on solutions.”

Projects in Pipeline

Cognitive Tutor was born out of work begun by John R. Anderson, a prominent cognitive scientist at Carnegie Mellon, in the 1970s. Mr. Anderson wanted to use the system to test a framework for modeling how advanced students process and learn information.

In 1998, though, Mr. Anderson, Mr. Koedinger, and their research partners had a string of products that were ready for mass distribution. They formed Carnegie Learning Inc., a Pittsburgh-based company, to promote Cognitive Tutor nationwide.

Meanwhile, other researchers were also experimenting with the technology and adding twists. Some of the other innovations in the production pipeline include:

Learn more about Project LISTEN and AutoTutor.

• Project LISTEN, a system developed by Carnegie Mellon researcher Jack Mostow that employs speech-synthesizing technology to “listen” to students read and give them feedback. The computer also assigns students new stories based on their progress.

Studies so far show that elementary school students who use the program in addition to their regular reading instruction for as little as 20 minutes a day show learning improvements equivalent to about half to two-thirds of a letter grade better than their regular-classroom counterparts.

• Online intelligent-tutoring systems in chemistry and applied mathematics that allow students to plug in their own homework problems, rather than rely on computer-generated tasks. Developed by Quantum Simulations Inc., of Murrysville, Pa., the chemistry programs, which are already on the market, have been tested with high school and college students.

The studies show that high school students who used the tutor at home were better at balancing chemical equations than were their peers who did not use the technology.

The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.
—Image courtesy of Institute for Intelligent Systems

Talking Head: The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.

• AutoTutor, a system developed by researchers at the University of Memphis, in Tennessee, and Carnegie Mellon that features an on-screen talking head that conducts a dialogue with students. Developed primarily for physics and computer-literacy studies at the university level, the program is now being tested with students in grades 8-11 as well.

“The goal is to help students reason,” said Arthur C. Graesser, the Memphis researcher heading that project. “When we compare it to just reading texts, we find that the AutoTutor helps by about one letter grade.”

Reading Body Language

An open question is whether the systems work as well in subjects whose subject matter is less specified, or more the focus of contention, than in math and science.

Experts also wonder how widespread such programs can become, given the time and expense they take to develop. Cognitive Tutor, for instance, was developed and refined over more than 20 years. Even now, researchers continue to tweak it.

“For every hour of instruction, there’s hundreds of hours spent in development,” said Mr. Anderson of Carnegie Mellon.

Yet Carnegie Learning has reached a point now where it can market its product for $30 to $60 per student and earn enough to keep devising new products, according to Steven B. Ritter, another of the company’s founders.

Researchers are also discovering, though, that intelligent-tutoring systems are as valuable as research tools as they are as teaching instruments. That’s because the systems can track student behaviors with a precision not available to classroom researchers, who rely on twice-a-year surveys or occasional classroom observations for their data.

With AutoTutor, Mr. Graesser says, researchers have embedded sensing devices to track students’ facial expressions, voice signals, and posture.

“We’re getting good at detecting boredom versus frustration versus confusion,” he said. “The more difficult the AutoTutor is, the better we get students to learn, and the less they like it.” That may not be so different, after all, from their experiences with human teachers.

Related Tags:

Coverage of education research is supported in part by a grant from the Spencer Foundation
A version of this article appeared in the April 04, 2007 edition of Education Week as New Breed of Digital Tutors Yielding Learning Gains


Student Well-Being Webinar After-School Learning Top Priority: Academics or Fun?
Join our expert panel to discuss how after-school programs and schools can work together to help students recover from pandemic-related learning loss.
Budget & Finance Webinar Leverage New Funding Sources with Data-Informed Practices
Address the whole child using data-informed practices, gain valuable insights, and learn strategies that can benefit your district.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
ChatGPT & Education: 8 Ways AI Improves Student Outcomes
Revolutionize student success! Don't miss our expert-led webinar demonstrating practical ways AI tools will elevate learning experiences.
Content provided by Inzata

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Achievement Opinion How Do We Measure Student Engagement? A School Created Its Own App to Find Out
The school-created app enables students to share their feedback on lessons and teachers to find out what students are saying.
Jana Benson
4 min read
Untitled design (1)
Student Achievement Biden Administration Wants College Students to Step In as K-12 Tutors
The Education Department released a letter encouraging colleges to let their students take work-study jobs as K-12 tutors.
5 min read
Image of two women working together at a table.
Student Achievement History Achievement Falls to 1990s Levels on NAEP; Civics Scores Take First-Ever Dive
Eighth graders' scores in U.S. history and civics dropped on the test known as the “Nation’s Report Card."
8 min read
Photo of students taking exam on laptops.
iStock / Getty Images Plus
Student Achievement Opinion Effective Summer Learning: ‘We Already Know How to Do This’
There’s no path to education equity that doesn’t include summer programs.
Chris Smith & Paul Reville
3 min read
conceptual illustration of 2 figures atop unequal boxes of summer experiences
Vanessa Solis/Education Week + Frances Coch/iStock + Getty images