A missionary and a cannibal cross the river together first, then a mission- ary returns alone. Next, two cannibals cross together, and a cannibal returns alone. Then two missionaries cross together, and a cannibal and a mission- ary return together. Then two mission- aries cross, a cannibal returns, then two cannibals cross, one returns, and the last two cross.
Some of the most logical puzzles are in logical terms, fairly simple, and would be easy for a com- puter to solve. But they are a challenge for the less logically minded human brain. The only way one statement on a box can be true while the other two statements are false is if the treasure is in box number two.)
Specialized cells called neurons, linked together in a web of infinite complexity, conspire to make the human brain the world’s pre-eminent insight machine. It learns from experience, compensating for its logical failings by mounting bold intuitive leaps. Not even the most sophisticated computer can match its awesome power. But a pioneering band of scientists, called “connectionists,’' are unmasking the mysteries of the mind, hoping to fashion a link with a revolutionary new kind of artificial intelligence, called neural networks--machines that “think’’ like the brain.
By William F. Allman
“Well, the first thing you see is the outside triangle, so you figure that it’s possibly that one because it’s real different on the outside. Then you realize that many of them are different on the inside.’' Jay McClelland, sitting in his office at Pittsburgh’s CarnegieMellon University, is trying to solve a seemingly simple problem. The 40year-old professor of psychology is looking at the figures shown below.
The question is, which of these figures is “most different’’?
“Let’s see,’' he says. “Well, the triangle is different. But why should differences on the outside be any more important than differences in the inside? So let’s look for some more differences. There are several: whether the outside is a circle or not, whether the inside is a circle, whether there’s a dot in the middle of the inside, and whether the inside is standard size. On that basis, I’d say that the first one is sort of the prototype, because it has the typical value on all the dimensions. And the second one differs on one feature, the third one differs on one feature, the fourth one differs on one feature, and the fifth differs on one feature. So on that basis I would say that I can’t make a decision.’'
This problem, designed by psychologist Peter Hayes, is tricky. The first figure, which McClelland calls the prototype, shares the most features with all the other figures. It is the least different of the group. But since all the other figures differ by one attribute, this “least different’’ figure is in fact the “most different’’ because it is the one that is most alike. On being told this solution, McClelland reacts the way most people do--with a Bronx cheer.
Put an index finger on each temple. Between your fingertips lies a threepound blob of biology that is an incredible machine. It does mundane regulatory tasks like keeping you upright and breathing. It does sophisticate motor tasks like guiding your fingers to get an egg from the refrigerator and crack it into a hot frying pan. It tells you what the world out there is like: picking up molecules in the air that indicate the toast is burning and sensing the airborne shock waves that tell you the cat is meowing to be fed. It synthesizes information, too: It knows that the objects on the kitchen shelves are glasses, not coffee mugs, and that the things in the sink are bowls, not plates. Your brain would probably know all this even if it were the first time it had seen these things.
Your brain doesn’t just process information; it also stores it in a number of ways. You can look at a number like 244-7628 and remember it long enough to get from a phone book to a telephone, but if the line is busy, you may have to look up the number again to re-dial. If 244-7628 is your best friend’s phone number, you probably won’t have to look it up at all. And if you think hard enough, you can probably still remember a phone number you had when you were a child.
The information the brain is capable of storing goes beyond simple numbers: a whiff of perfume, the smoke from a pipe, the smell of hot tar in an August afternoon, the waft of waffles-- all can cause your brain to spill reels of memories of a loved one, a cherished ritual, or even just a simple activity from long ago. Often you remember more than visual scenes; the brain can evoke the eerie sensation of the way you felt, too. Think of one of your most embarrassing moments; even though it was long ago, you may still feel an uneasiness in your stomach.
The mind also creates new ways of manipulating the environment around it--albeit not always for the better. The problems it solves go beyond simply building better and better mousetraps. In fact, it tries just about anything. The same brain that invented the wheel and the digital watch also tries to solve less practical puzzles such as How did the universe begin? or What is morality? In fact, this same brain actually enjoys doing problems just for fun--puzzles such as Which of these shapes is most different?
How do our brains do all these things? For the last three decades, most cognitive scientists have researched this question under a basic assumption: If you observe the human brain in action long enough, you’ll notice that it displays some kinds of regular, mechanistic behavior that can often be characterized as a set of rules. But unlike auto mechanics, brain scientists can’t examine the machinery of the mind by looking under the hood. So instead, they “test-drive’’ the brain by giving it puzzles.
Some of the puzzles that brain researchers ask their subjects to solve are not really puzzles in the ordinary sense. Many are so simple that people do them almost instantaneously and without consciously thinking. In one type of study, for example, subjects sitting in front of a screen are presented with pairs of letters such as aa or ac. They are asked to push one of two buttons, indicating whether the two letters are the same or different. It sounds pretty easy, but when, for example, the pair Aa is presented, it takes longer to decide whether the example contains the same letters than when aa is shown. Participants are not consciously aware of the delay, and it is not much, less than a tenth of a second, but it happens repeatedly. This “reaction time’’ (how long from the presentation of a problem to the solving of it) can be a clue to what’s going on in the brain.
While there are reams of reactiontime data, psychologists can only speculate what is going on inside the “black box’’ of our brains while it solves these kinds of problems.
Therefore, some psychologists use other kinds of puzzles to try to find those rules that govern the mind’s machinery. Subjects are given a puzzle and asked to say out loud what is going on in their minds while they are trying to solve it--just what McClelland was doing with the “most different’’ problem. In one experiment, for example, a student was asked to say what was going through his mind as he solved this cryptoarithmetic problem:
Knowing that D=5, what numbers do the other letters in this problem represent? (Answers on page 29)
DONALD
GERALD
ROBERT
The student’s comments while he solved the problem, detailed in a book by computer researchers Allen Newell and Herbert Simon, Human Problem Solving, ran some 15 pages. To solve the problem, the student showed a variety of rulelike behaviors: setting goals, making assumptions and testing them, and rejecting hypotheses that didn’t work.
Using similar types of rules, Newell, Simon, and Cliff Shaw created a computer program called the general problem solver, or GPS. It could look back at its performance and look forward to its goals, generating possible strategies and testing them. It could prove theorems in formal logic, do trigonometry, and even solve puzzles similar to this one:
Three cannibals and three missionaries are on one side of a river and need to get to the other side. There is a boat at the bank of the river, but it is small and can only hold two people at a time. How do you get all the cannibals and missionaries across the river without ever allowing the cannibals waiting on either side of the bank to outnumber the missionaries?
The GPS was very powerful because a problem like cannibals and missionaries could be solved using rules, in this case the rules of logic.
This rule-based model of the mind was extremely appealing to artificialintelligence researchers and psychologists. Computers are designed to be run by rules, making them perfect for testing rule-based theories of how the mind works. To psychologists, exploring the mind of a “higher’’ level of rules seemed a plausible way to avoid having to consider the complex interaction of the brain’s billions of neurons.
Plausible as this approach might appear, however, McClelland and the other connectionists are trying to replace this rule-driven description of our thinking processes with a new model of the mind that incorporates the structure of the brain itself. In the neural network model of the mind, thinking is not following a set of rules but a product of the complex interactions of huge numbers of neurons.
One of the primary reasons for connectionists’ rejecting making rules to approximate the activity of the mind is the same reason rules became popular in the first place--the regularity of our behavior. At some level, it does appear that we behave in a rulelike manner. But if you look closer, you see quite the opposite; it seems that we regularly use anything but rules to get by in the cognitive world.
Instead, we appear to use a process closer to intuition. For example, most of us have been doing “most-different’’ problems since the first time a child psychologist waved an intelligence test under our noses. Since the problem shown at the beginning used a similar language, format, and shapes to other puzzles, McClelland naturally assumed it was no different. You could almost say that he actually behaved brilliantly because he took cues from the way the problem was presented, immediately generalized them to other problems he had seen, and realized what kind of information he needed in order to find a solution. This generalization process is a large part of solving any problem.
There are other problems more brazen in their challenge to our brain’s power of assumption: How many types of animals did Moses take on the ark? If a plane crashes on the U.S.-Canadian border, in which country are the survivors buried? Is it legal for a man to marry his widow’s sister?
These problems are mostly designed not for the entertainment of the solver, but for that of the asker who can smirk while you struggle to “solve’’ the problem instead of realizing that Moses didn’t go on the ark, that survivors don’t get buried, and that a man who has a widow is dead.
Much of our mind’s power comes from the fact that it’s a good guesser, and problems like these turn that power upside down. Trick problems would be easy for a computer because they are basically asking just for factual information with a red herring thrown in. A computer isn’t tricked by trick questions because, in a way, it isn’t smart enough.
These kinds of problems are relatively rare because, like judo masters, they use the great strength of their opponent--the brain--to their advantage, and that requires some skill. Also, such problems may be rare because there is a far easier way to throw a banana peel in the brain’s lumbering path. (See the figure below.)
If one, and only one, of the inscriptions on the boxes is true, which box should you open to find the treasure?
The reason these types of puzzles are so numerous--and so difficult for us--is that they go straight for the Achilles heel, as it were, of the brain: using logic.
According to McClelland, being intelligent goes beyond simply doing logic or recognizing patterns. Instead, the essence of high-level thinking is making generalizations from past experiences and applying them to new situations. “What’s smart is having insights,’' says McClelland. “Being able to figure out what to do in novel circumstances. There’s no such thing as insight in a computer program like Simon and Newell’s General Problem Solver.’'
Producing a new response from previous experiences enables neural networks to do some kinds of cognitive feats without explicitly using rules. McClelland and David Rumelhart, a connectionist at Stanford University, designed a neural network that learned to change the present-tense form of a verb to its past-tense form. Rather than being programmed with rules for constructing the new form of the verb, the network was trained by being shown a group of verbs and their past tenses.
The network memorized irregular verbs--changing see to saw, for example. But the network was also able to learn various overall patterns for regular verbs-- changing guard, for example, to guarded. Though it was designed to do so, this neural network made the same kinds of mistakes that children make when they are learning verbs. For example, at first the network would learn that an irregular verb such as go changes to went. But as the network continued learning and assimilated the patterns of regular verbs, it went through a period of changing go to goed.
Once the network was fully trained, it knew how to transform both regular and irregular verbs, even those it hadn’t seen before. “The network isn’t just recognizing a word it has learned,’' says McClelland. “It’s synthesizing a new response. It can give you the correct past tense of completely new words. Even when given a made-up word such as grok, for example, the network will produce grokked. It learns from experience and generalizes.’'
Rumelhart and McClelland’s network is a very simple model, and not intended to be a complete representation of how children acquire their language ability. But it does provide a demonstration that sophisticated tasks can be performed by a neural network that learns to generalize from examples rather than using logical rules. Perhaps the ability to generalize from experience, not logical ability, is what makes our brains so good at understanding speech and recognizing faces. The ability to draw on previous experiences to respond to new situations makes our brain, like a neural network, an insight machine, while the serial rule-and-symbol processing of a conventional computer makes it an ideal logic machine. “Connectionism not only accounts for our weaknesses in doing logic problems and the like,’' says McClelland, “but it also accounts for our abilities to be much better at thinking than anything else is--including machines.’'
Such irrationality may also provide the glue that holds society together, such as our compassion for people we don’t know or may never see again and our unwillingness to do something because “it doesn’t seem right,’' even though it may be a strictly logical move for other reasons. Our irrationality may also account for economists’ problems in making forecasts; economists often assume that people will behave logically when making financial decisions. Ultimately, our irrationality may account even for the fact that we are able to fall in love, despite what reason might say about the fate of many such relationships.
In fact, our irrationality, along with its good and bad consequences, is the trait that distinguishes us from every other creature--and machine--on the planet. It, and not our ability to do logic, is what makes us human. Our blundering, irrational brain creates our hatred and bigotry, envy and paranoia, pride and greed. But it is also behind our ability to enjoy music, forge a sense of justice, believe in things we can’t see, and empathize with strangers. It’s the source of hope, love, the movies of Charlie Chaplin, and our ability to get out of bed in the morning with the knowledge of our mortality.
William F. Allman, a senior editor at U.S. News & World Report, has written for The Washington Post, Esquire, and Technology Review. His previous book, Newton at the Bat, a collection of essays on sports and science, was selected as one of the Best Books for Young Adults by the American Library Association.
