Until now, most research on technology’s effectiveness has taken the form of small case studies, some of which examined just a classroom or two at a time. Though several of these studies found evidence that technology can improve student achievement, the findings don’t necessarily apply to other classrooms where teachers are less motivated or knowledgeable about computers.
A new national study on the effectiveness of teaching with technology appears to confirm what many educators have suspected for years: Computers can raise student achievement, but they can also do more harm than good when used the wrong way.
“Technology, indeed, can have positive benefits,” says Harold Wenglinsky, a researcher at the Princeton, New Jersey-based Educational Testing Service who carried out the study for Education Week, a sister publication of Teacher Magazine. “But those benefits depend on how the technology is used.”
Wenglinsky has broken new ground by analyzing a national database. His study, “Does It Compute? The Relationship Between Educational Technology and Student Achievement in Mathematics,” is based on data from the 1996 National Assessment of Educational Progress--specifically the math scores of 4th and 8th graders. Administered by the U.S. Department of Education, the NAEP has tracked student achievement for nearly three decades. In 1996, it asked students and teachers for the first time how they used computers in math.
After factoring out the influence of variables that might affect achievement, such as students’ socioeconomic status and class size, Wenglinsky found strong links between certain kinds of technology use and higher scores on the NAEP. Some key findings:
* Eighth graders whose teachers used computers mostly for “simulations and applications"--both of which are associated with higher-order thinking--performed better on the NAEP than students who had not received that kind of instruction.
* Eighth graders whose teachers used computers for mere “drill and practice” performed worse than other students.
* Fourth graders whose teachers used computers mainly for “math learning games” scored higher than others.
* Students who spent the most time on computers in school did not score any higher than their peers; in fact, they performed slightly worse.
* Low-income and black students are the most likely to have teachers who do not use the new technology to its full advantage.
For years, educators have sparred over the use of computers in school. Proponents portray the machines as something of a cure-all for American education, while critics argue that they add little to the classroom and may even be a hindrance. Wenglinsky’s study validates elements of both views. What matters most, it suggests, are not computers themselves but what teachers and students do with them. “People are getting beyond the idea that this thing is magic, and that, like fire, just by sitting near it you can get some benefit,” says Christopher Dede, a professor of education and information technology at George Mason University in Fairfax, Virginia.
What Wenglinsky found is that the educational benefits of computers are greatest when they are used in sophisticated ways. For example, 8th graders whose teachers used computers primarily for simulations and applications scored two-fifths of a grade level higher on the NAEP than other students. Simulation programs illustrate relationships and allow students to test the effects of changing variables; application software, such as spreadsheet programs, lets students manipulate and analyze data.
“The computer’s most powerful uses are for making things visual,” says James Kaput, a math professor at the University of Massachusetts at Dartmouth. “It can make visual abstract processes that are otherwise ineffable.”
Though Wenglinsky identified no discernible effect of using computers for simulations and applications among 4th graders--probably because few elementary teachers use technology for this purpose--he did find that 4th graders whose teachers used computers for math learning games scored higher on the NAEP than their peers.
“Students are motivated when they can influence the outcome of the activity,” says Douglas Clements, an education professor at the State University of New York at Buffalo. “And games have the benefit of providing immediate feedback.”
Wenglinsky’s study also found that effective use of technology can enhance a school’s “educational” climate. Schools, for example, where teachers were using computers for more sophisticated ac- tivities than drill and practice were more likely than others to report high attendance, low tardiness, and better morale.
“Technology certainly can be highly motivating,” says Barbara Means of the Center for Technology in Learning at SRI International. “Teachers see it in their kids’ interest. They see them come in at lunch and after school to work with it.”
Technology also can liberate teachers from traditional lecture-style instruction, opening the door for more progressive forms of pedagogy. “One of the real benefits of different types of technology,” Wenglinsky says, “is the way they influence how teachers and students relate to each other.”
In fact, a growing number of technology advocates argue that this is the chief benefit of computers in the classroom. “Kids learn by doing, by presenting, by displaying, by engaging,” says William Fiske, educational technology specialist at the Rhode Island department of education. “Learning happens best when the youngsters are doing the heavy lifting.”
Unfortunately, most of the nation’s schools aren’t using computers in ways that Wenglinsky’s findings indicate are linked to better learning. And survey responses from the 1996 NAEP also raise serious questions about technology’s role in closing the achievement gap between disadvantaged students and their peers.
Over the past decade, tremendous resources have been committed to ensuring that schools in disadvantaged communities have computer equipment and networking capabilities on par with more affluent areas. Survey results show, for example, that black students use computers in learning math more often than white students.
But the survey paints a very different picture about how those students use computers. In 8th grade, about 31 percent of white students used computers mostly for simulations and applications, compared with 14 percent of black students. More than half the black students had teachers who used computers mostly for drill and practice, compared with only 30 percent of white students.
In short, black students have closed the digital divide where it matters least--the amount of time on a computer. The gap persists where it matters most--how the computer is used. “It’s the low-income communities that have invested the most in technology for drill and practice,” says Margaret Honey, deputy director of the New York City-based Center for Children and Technology. “But if this research says those investments don’t have the kind of impact we want them to have, then that’s an important message. There’s an expectation among many urban administrators that kids from disadvantaged backgrounds need basic skills and nothing else.”
The study offers more encouraging findings in the area of professional development. Teachers who had received any technology training within the past five years were more likely than those who hadn’t to use computers in ways that Wenglinsky found to be effective. And 8th graders whose teachers had received training performed more than a third of a grade level better than peers whose teachers had no training. The training was also linked to gains at the 4th grade level, although the difference in student scores was smaller.
Wenglinsky was struck by the fact that any amount of professional development with computers translated into student achievement gains. This, he says, “leads one to think that more elaborate training might post even greater gains.”
The ETS researcher knows that his findings only scratch the surface and that there is much more to learn about the link between computers and student achievement. His study, after all, examined only a single year of data and one field of study, math.
Still, Wenglinsky can now tell educators something they’ve been waiting a long time to hear. “What we know for certain,” he says, “is that when teachers use the computer to teach higher-order thinking skills, students benefit.”
--Jeff Archer
