School & District Management

Data Mining Gets Traction in Education

By Sarah D. Sparks — January 11, 2011 5 min read

The new and rapidly growing field of educational data mining is using the chaff from data collected through normal school activities to explore learning in more detail than ever, and researchers say the day when educators can make use of Amazon.com-like feedback on student learning behaviors may be closer than most people think.

Educational data mining uses some of the typical data included in state longitudinal databases, such as test scores and attendance, but researchers often spend more time analyzing ancillary data, such as student interactions in a chat log or the length of responses to homework assignments—information that researchers call “data exhaust.”

Analysis of massive databases isn’t new to fields like finance and physics, but it has started to gain traction in education only recently, with the first international conference on the subject held in 2008 and the first academic journal launched in 2009. Experts say such data mining allows faster and more fine-grained answers to education questions that ultimately might change the way students are tested and taught.

“Data resources you wouldn’t necessarily think would be useful can turn out to be very powerful for making inferences,” said Ryan S. J. d. Baker, an assistant professor of psychology and learning sciences at Worcester Polytechnic Institute in Massachusetts.

For example, research from the Pittsburgh-based Carnegie Mellon University found small changes in the length of time a student took to answer individual test questions signaled the student was struggling, cheating, or had given up in favor of filling in answers randomly.

Expanding Data Universe

In centers such as the Pittsburgh Science of Learning Center’s DataShop, researchers use advanced computers to analyze 238 data sets of online and classroom data, comprising 49 million individual student actions.

“You might be collecting thousands of data points for a single student—in some areas virtually millions—whereas the traditional qualitative methods in education psychology might have dozens or even a hundred measures,” said Arthur C. Graesser, a psychology professor at the University of Memphis and editor of the Journal of Educational Psychology.

These data haven’t been studied in such depth before because it’s only possible to find significant results when researchers can study a huge number of data points. For example, Mr. Baker studied a topic that has frustrated teachers for generations: students who try to get through a task without actually learning the material.

“Students spend on average 3 percent of the time gaming the system; maybe 15 [percent] of students will do it at least once,” Mr. Baker said. With only a few dozen students, it’s almost impossible to tell exactly when and how it happens, he explained, “but when you have data from thousands of students, you can.”

Studying hundreds of thousands of data points on students working through an online tutoring program, Mr. Baker created a program to recognize when a student was attempting to complete a task without mastering the material, and then present the missed material again in a new way.

Research that draws on educational data mining may also compress the lag time between undertaking a study and getting usable results, addressing a common critique from educators.

“In the past, somebody runs an efficacy study where they spend five years trying to study a sample that may include more than one classroom, and it takes a lot of time and a lot of money,” Mr. Graesser said. “whereas EDM [educational data mining] study provides a far richer set of data on students in a matter of weeks or months. It’s a whole different style.”

States Experimenting

For practicing educators, the question educational data mining raises is: Does this mean researchers could create tools for teachers that collect information in the same way that Amazon.com, the online retailer, collects information on customers’ buying habits? Could systems be developed that can track whether a student is excited about some topics but not others, or struggling with decimals but not long division, and suggest interventions accordingly?

“Oh yeah, no problem! We have done that already,” said Greg Chung, the co-principal investigator of the Center for Advanced Technology and Schools at the University of California at Los Angeles. In the early 2000s, his team developed a program for the U.S. Marines that identified which soldiers were likely to have trouble with different aspects of marksmanship based on their understanding of trigger control and then automatically assigned soldiers study materials. By the end of one week on the program, the participating Marines developed better marksmanship skills.

Mr. Chung and other researchers said the technology and research can be developed faster than it takes to teach practitioners how to use it.

Mr. Chung recalled giving teachers electronic clickers that would allow every student in a class to answer a question—as opposed to only two or three in a classroom—and would allow the teacher to analyze their responses. But the sudden flurry of responses—and their range—quickly overwhelmed the teachers. “The teachers said, ‘Yeah, this is interesting, this is cool, and we learned a lot about our students, but what do you do in a class with so many different levels?’ ” Mr. Chung said. “They couldn’t address every kid.”

Several states, including Louisiana and New York, are experimenting with data tools that allow teachers and principals to track daily attendance, behavior and academic performance of each student.

In fact, a 2009 study by a team of researchers from Carnegie Mellon and Worcester Polytechnic found in the process of creating an online tutoring program that its underlying data model for tracking student progress could predict students’ year-end academic performance better than scores on the state’s standardized test.

“If we could show that a student’s work over time was a better predictor of student success than these state exams that everyone complains about anyway, wouldn’t that help us get a lot farther along?” said John C. Stamper, a systems scientist in the Carnegie Mellon Human-Computer Interaction Institute and technical director of the DataShop.

A version of this article appeared in the January 12, 2011 edition of Education Week as Data Mining Gets Traction in Education

Events

Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
How Principals Can Support Student Well-Being During COVID
Join this webinar for tips on how to support and prioritize student health and well-being during COVID.
Content provided by Unruly Studios

EdWeek Top School Jobs

Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Arizona School Data Analyst - (AZVA)
Arizona, United States
K12 Inc.
Software Engineer
Portland, OR, US
Northwest Evaluation Association

Read Next

School & District Management New York City's Equity-Minded Schools Chief Resigns
Richard A. Carranza, the chancellor of the New York City schools, announced Feb. 26 he will step down from the job next month.
4 min read
Richard Carranza, Chancellor of the New York City Department of Education, arrives to Public School 188 The Island School as students arrive for in-person classes, on, Sept. 29, 2020, in the Manhattan borough of New York.
Richard A. Carranza announced he will depart the top New York City schools job in March.
John Minchillo/AP
School & District Management Opinion New Resource Tracks School System Reopening
The Return to Learn Tracker identifies the current instructional model of all regular public school districts with three or more schools.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School & District Management San Francisco School Board Pauses Renaming 44 Schools, Promises to Consult Historians
The renaming of 44 schools in the San Francisco Unified School District is apparently being put on hold after intense blowback.
Greg Keraghosian
1 min read
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.<br/><br/>
Jeff Chiu/AP
School & District Management Superintendent Who Led During COVID-19 School Shutdowns Gets Top Honors
Michelle Reid of Washington state's Northshore district, one of the very first to close schools last March, was named National Superintendent of the Year.
3 min read
Michelle Reid, superintendent of the Northshore district in Washington
Michelle Reid, the superintendent of the Northshore district in Washington, was named National Superintendent of the Year.
courtesy of AASA, the School Superintendents Association