School Researchers Use Computer Models to ‘Field-Test’ Ideas

By Sarah D. Sparks — October 28, 2014 5 min read
  • Save to favorites
  • Print

What makes one intervention work in a school when another seemingly similar one falls flat?

Increasingly detailed computer models of student behavior and learning may help researchers avoid such setbacks by better pinpointing interventions before taking them to schools.

“In education research, I get a great idea, apply for funding, … then I spend a few months in schools taking time from students and teachers, and often find out it doesn’t work,” said Richard L. Lamb, an assistant professor of science education and educational measurement at Washington State University in Pullman.

“That’s great that we have that data,” he said, “but it’s not the most efficient way to do [research and development].”

Instead, Mr. Lamb and colleagues are working to pair education technology and neuroscience to mimic how students learn in a classroom and provide an additional means of testing and honing interventions.

Building Virtual Classrooms


A Washington State University project is developing a virtual student pool for researchers to test interventions by tracking real students performing cognitive tasks in video games.

Students play video games that include tasks shown to require specific cognitive functions.

The artificial intelligence records what a student does and learns to approach a problem as the student would.

The system can be used to simulate hundreds or thousands of students.

Researchers can use the simulations to supplement school-based field testing of an intervention.

SOURCE: Education Week

The Student Task and Cognition Model, or STAC-M, is an “artificial neural network,” a type of artificial intelligence system that mimics human learning and pattern recognition.

The project so far has collected data on more than 2,000 high school students ages 14 to 18.

While the main group is nationally representative, Mr. Lamb said the researchers are also planning to sample specific student groups, such as students of different language backgrounds.

The Washington State University project is part of a trend of increasing use of online and live student-data collections during early phases of education research.

Kenneth R. Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon University, in Pittsburgh, who supervises a separate student-data-analysis initiative but who is not part of the Washington State University project, said education watchers are seeing a steady “trickle” of new projects creating more dynamic uses of education data.

He believes artificial learning networks will be a normal part of education research in a decade or so.

“I think that will increasingly be how we can and should evaluate new approaches to instruction,” Mr. Koedinger said.

Teaching a Machine ‘Brain’

For example, a model like STAC-M, which can simulate the effects of an intervention on 100,000 students, could supplement field-testing of a new program in real schools to produce a larger overall sample and strengthen the reliability of the study’s findings.

In the first STAC-M pilot, which is depicted in the 2014 study, Mr. Lamb and his colleagues asked 645 high school students from a Mid-Atlantic traditional public high school to play science-based video games in the lab.

Several of the game’s tasks required critical science reasoning and understanding of concepts like conservation of mass, volume, liquids, and numbers.

In previous studies using functional magnetic resonance imaging, the tasks had been associated with activation in specific parts of the brain involved with reasoning and executive function.

Fifteen hours of play created more than 450,000 data points.

“Kids do lots of things in games, and with game technology, you can get lots of background: what they click on, what they answer, what they don’t answer, what tools they use,” Mr. Lamb said.

Each student’s responses “taught” a part of the system to approach and solve problems in the way the student would.

Just as adaptive tutoring systems “learn” how to teach the next lesson from a student’s current responses, the STAC-M creates a population of virtual students who respond in the same way live students would.

Not only can it respond to changes in similar science tasks, but also to other problems that would use the same type of cognitive skills.

For Mr. Koedinger, the artificial-student system starts off learning much more slowly than real students do, but as it progresses, it tends to learn a bit faster than live students. In part, he thinks that might be because the artificial intelligence is “much more logical” than a student. “A human can consider and reconsider uncertainties—take certain things as uncertain when they really are certain,” Mr. Koedinger said. “The [artificial] system takes some things for granted more than the human brain does.”

But that’s where students help teach the system.

A computer learning how to do a science task won’t second-guess itself or forget what it previously learned because it is worried that someone else won’t think it is good at that particular subject—but it can learn to replicate that slowdown in function when modeling hundreds of students over time.

“The data gets you to a place you wouldn’t get to otherwise; it’s already discovering elements of real student learning that have implications for educational design,” Mr. Koedinger said.

Tweaking the Testing Pool

After collecting the data, the researchers were able to simulate a separate intervention of 100,000 students, varying the curriculum and students’ levels of initial background knowledge to test different ways of teaching critical thinking as either an independent skill or one that is taught as part of a specific subject.

Mr. Lamb and his colleagues suggested that pairing the system with future brain-imaging studies could help researchers pinpoint the ways in which students respond to very similar-seeming interventions differently at the neurological level.

“I’m not saying we never have to go into the classroom again, but we can run the model to target before we go into the classroom,” Mr. Lamb said.

Challenges in Systems

However, Cynthia Coburn, a professor of education and social policy at Northwestern University, in Evanston, Ill., and a veteran of long term field-testing in schools, warned that data simulations should never take the place of full-scale experimental trials in schools, because interventions often succeed or fail for reasons that have nothing to do with how students learn.

“The history of education research is littered with really wonderful lab experiments and interventions created outside the classroom that, for whatever reason, didn’t work in the classroom,” Ms. Coburn said.

“I think, bottom line, what we’re trying to understand are phenomena of student learning, teacher learning,” she added. “Those processes fundamentally occur in schools, so you’re never going to get anywhere if you don’t look at them in schools.”

Mr. Koedinger of Carnegie-Mellon agreed: Rather than artificial student models replacing live testing in schools, he said he envisions trained teachers and researchers in every school eventually using ongoing computer modeling and data analysis to set up natural experiments during the regular course of school operation.

“The false premise is that before we do an intervention, we have to be sure” it works, Mr. Koedinger said, but, “there are experiments going on in schools all the time—variation in the textbooks, teachers, etcetera—but there’s not monitoring of those choices, no observations to see on a regular basis whether these things work. That’s what we need.”

A version of this article appeared in the October 29, 2014 edition of Education Week as New Computer Models Help Researchers ‘Field-Test’ Ideas


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Data Webinar
Drive Instruction With Mastery-Based Assessment
Deliver the right data at the right time—in the right format—and empower better decisions.
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Teaching Profession Webinar
How Does Educator Well-Being Impact Social-Emotional Awareness in Schools?
Explore how adult well-being is key to promoting healthy social-emotional behaviors for students. Get strategies to reduce teacher stress.
Content provided by International Baccalaureate
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
IT Infrastructure Webinar
A New Era In Connected Learning: Security, Accessibility and Affordability for a Future-Ready Classroom
Learn about Windows 11 SE and Surface Laptop SE. Enable students to unlock learning and develop new skills.
Content provided by Microsoft Surface

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Data Opinion How to Start Using Data to Achieve Equity for Students
Schools are awash with data but rarely do anything with it. One district started out by limiting the tools it uses.
Mackey Pendergrast & Erica Hartman
5 min read
Entry Point
Data Education Department to Expand Data Collection on COVID and Schools
The Education Department will expand the limited survey started last year to query about issues like learning loss and health precautions.
2 min read
Image of a data dashboard.
Suppachok Nuthep/iStock/Getty
Data 'Extreme' Chronic Absenteeism? Pandemic School Attendance Data Is Bleak, But Incomplete
Only a handful of states have reported information on chronic absenteeism, right as advocates warn that addressing it will be a top priority this fall.
4 min read
Image of an empty desk.
Laura Baker/Education Week and kazuma seki/iStock/Getty
Data Knowing What Schools Did in the Pandemic is Crucial. So Is Preserving That Data
A prominent researcher is working to collect information on schools and COVID-19 to inform future study on the pandemic's effects.
6 min read
Kindergarten students wear masks and are separated by plexiglass during class at Milton Elementary School, in Rye, N.Y. on May 18, 2021.
Kindergarten students wear masks and are separated by plexiglass during class in May at Milton Elementary School, in Rye, N.Y.
Mary Altaffer/AP