For one blissful week this summer, Kimberly Hooverson settled down in an apartment on the outskirts of Seattle, a television remote control in her hand and a healthy skepticism in her heart, to tell one of the world’s most successful companies what it is doing wrong.
It was heady stuff for a 12-year-old.
Professional ethics, not to mention the admonitions of product-testers at the Microsoft Corp., prevent her from discussing exactly what it was she was critiquing for the Redmond, Wash.-based software giant. But Kimberly can say she was watching what appeared to be an interactive television program.
“They wanted us to explore their software,” she explained. “They wouldn’t give us instructions on how to use it because they wanted to see how easily we adapted to it. It was kind of hard at first, but we got used to it.”
The job, she says, suited her, and her friend Amanda Forsell, just fine. Both girls are 6th graders at Centennial Middle School in Snohomish, Wash.
As a software tester, Kimberly joined the ranks of an elite group of youngsters across the nation who help software companies decide which products and features will fly and, perhaps more important, which won’t, in the highly competitive market for educational and “edutainment” software.
In some cases, whole schools serve as test sites. And even teachers are now being recruited to evaluate software before it goes to market.
Officials of Microsoft, mecc, Broderbund Software Inc., and other competing companies claim that one of the best ways to find bugs in software is to let potential users try it out. And few users, they aver, are as critical as children.
“Kids look at things differently than adults,” says Eileen Simpkins, a product manager in Microsoft’s consumer division. “And like it or not, we here haven’t been kids for a while.
“Even if you could remember what it was like to be a kid,” she adds, “my experience as an 11-year-old or 12-year-old was a lot different than it is today.”
Welcome Reviews
Software companies use a variety of strategies to solicit the opinions of prospective users.
The Lightspan Partnership Inc. of Carlsbad, Calif., which has just launched an interactive education service linking schools and homes over cable-television networks, allows both children and adults to preview and comment on its products at a network of test sites nationwide. Children also informally test competitors’ products in a “game room” at the company’s headquarters.
“We definitely have various levels of testing,” says Amanda Smith, who works for the company’s marketing division. “Maybe at the initial stages, we’ll bring in kids to sort of get general ideas about the product. It’s everything from the educational content to the motivational aspects of the product. We ask them, ‘Does this make sense?’ ‘Do you like this music?”’
But, she adds, there’s an added benefit: “It’s also just great to have kids around the office so that we are immersed into ‘kid culture.”’
Broderbund also uses children to test all its educational software, both at its Novato, Calif., headquarters and in local schools.
“Where we do it really is governed by what the product needs are and what assumptions we are making,” says Laurie Strand, the publisher for the company’s early-learning exploration group. “One of the things about going to the schools is that it’s helpful to have kids in their own environment. They have other diversions that compete with what you’re showing them. You get a better read of that out in the field.”
And testing is often extensive. At Microsoft, Simpkins estimates that a soon-to-be released product that allows children to create three-dimensional animations was tested by more than 300 children for more than 1,600 hours.
Christy Clark, the principal at the nearby Montlake Elementary School, where some of the test trials were conducted, welcomed the chance to take part. “The kids were real enthusiastic about it,” she says.
Microsoft testers visited the school testing sites once a week for several months when the product was under development. In return, participating schools received free software.
Despite the advantages of getting a child’s point of view, individual testing is only one component of creating marketable software. “You definitely have to filter the kind of information you get back from kids, or from anyone,” Strand concedes. “Yes, you want to check in with kids, but you just can’t have them saying ‘I don’t like the color of that character’s shirt.”’
Valued Opinions
Kimberly’s mom, Helen Lewis, recruited her as a software tester. She had seen a flier from Microsoft soliciting volunteers at the Safeco Insurance Co. where she works. Lewis, who herself was scheduled to test some software for Microsoft last month, says her daughter was enthusiastic about the project because playing video games “is one of her favorite things in life to do.”
The pay for children who test software generally involves merchandise gift certificates, T-shirts, or free software, but there are other less tangible benefits, too.
Whenever Caitlin Somero sees a copy of mecc’s Oregon Trail II in a software store, the 12-year-old student at Central Middle School in Columbia Heights, Minn., has the satisfaction of knowing she helped design the latest version of the 25-year-old software standard. Caitlin served as a member of mecc’s student advisory board.
She enjoyed working on the product because, unlike a lot of children’s software, it stresses problem-solving and interaction with the characters over action. “Some girls are into, like, violence, but I’m not,” Caitlin says.
Unfortunately, her friends were not all that excited about her job. “I told a few of them, and they didn’t think it would be neat because they thought it would be looking at unfinished products with no color,” she says. “But a librarian at my school thought it was really cool.”
Caitlin says she’s also intrigued by how sophisticated the process of developing software has become. “I thought it was just a few people,” she adds. “I didn’t know how many people it would take to do just one scene.”
Sharon Somero, Caitlin’s mother, added that the experience was a big boost for Caitlin’s self-confidence. “There were only five or six kids in the group, and she was one of them,” she says. “She got the feeling that the people at mecc really listened to the kids.”
Mecc has also recruited local technology-using educators to evaluate software products, while a separate group of teachers is evaluating the company’s Internet-based products.
“The folks who are participating feel their opinions are valued, which they are,” says Peter Contino, mecc’s product-line manager. “We said, ‘Hey, you’re going to be evaluating products that thousands of teachers are going to be using.”’
