Blended Learning Research Yields Limited Results

Blended learning is gaining considerable popularity in American classrooms, but the question remains: Is there strong evidence that the strategy helps K-12 students?

“The answer right now is still no,” said Sarojani S. Mohammed, a partner and lead researcher at The Learning Accelerator, a Cupertino, Calif., nonprofit group that helps districts implement blended-learning strategies. “We don’t have definitive evidence that blended learning works or that it doesn’t, though we do know some things about specific aspects.”

Blended-learning practices have steadily evolved in classrooms, but there is little consensus on what, exactly, the term encompasses. This further hamstrings efforts to build a solid understanding of whether, when, and how the strategy of combining face-to-face instruction with technology-based lessons actually works.

Research on blended learning has begun to accumulate only in the last few years, with the U.S. Department of Education, the Bill & Melinda Gates Foundation, the Michael & Susan Dell Foundation, and others having recently supported studies of its uses in classrooms.

“Whether blended learning works or not is a frustrating question because the answer is always going to be ‘it depends,’ ” said Michael B. Horn, a co-founder and the executive director for education at the Clayton Christensen Institute for Disruptive Innovation, in San Mateo, Calif., which studies technology in society. “Depends on how it’s implemented, how well teachers are trained. ... It’s unlikely to be that blended learning magically causes better learning, and more likely, that it offers better opportunity to provide each student with what he needs when he needs it.”

Even defining “blended learning” has proven difficult.

Terms go in and out of fashion—hybrid learning, virtual learning, technology-based instruction, personalization, and so on—and can describe virtually identical or radically different instructional models with disparate levels of technology use.

“Blended learning should be defined as a continuum,” said Susan D. Patrick, the president and chief executive officer of the International Association for K-12 Online Learning, based in Vienna, Va. “It’s really important for us to realize blended learning is not only a combination of online and face-to-face learning, but that students have some control over time, place, path, and pace.”

Coming to Terms

In a 2014 report underwritten by the Michael & Susan Dell Foundation, the research group SRI International studied 13 low-income charter schools using a “rotation” model of blended learning, in which students move among online and in-person stations for different parts of the school day. The SRI researchers who conducted the study found all the schools provided a formal education program with at least some online instruction and some coursework delivered outside the home, and students had some level of control over how they went through the material.

But even within a single model, blended learning looked different from school to school. In some cases, teachers had a big say in which programs students used at different times, while in others, in-person teachers had little connection to the separate labs where students worked online. In still others, self-directed online programs were more likely to be an enrichment for advanced students, while other schools focused on remediation for struggling students.

“In a traditional environment, you’re assuming a teacher is teaching the same thing at the same time, so you can code what’s happening [in an observation study]. That’s much harder with personalized learning,” said Ms. Patrick. “It’s hard to make general statements about blended learning when 20 different schools have 20 different models.”

Ms. Mohammed noted that there are “lots of different flavors” of blended learning,” and “the ecosystem has not really congealed around a definition.”

Parsing Imperfect Measures

And even when studies are all looking at the same learning model, that research is often focused on adults, not school-age users.

A 2010 federal study underscored the scope of the challenge. The Education Department took a broad look at all available studies of online learning, including blended approaches. From 1996 through 2006—the decade in which the Internet rapidly evolved and became ubiquitous—there were no experimental or controlled studies at all comparing online and in-person instruction for K-12 students.

The analysis ultimately found that students in blended-learning classes outperformed those in fully online or fully in-person classes and spent significantly more time on task. But it also found no significant improvements for K-12 students, for good reason: Out of 46 studies included, nearly all examined college-age or adult professional students.

Only five studies covered K-12 education, and they examined academic gains in a variety of subjects and age groups—8th grade social studies, 8th and 9th grade algebra, middle school Spanish, elementary special education—and, for a group of schools in Taiwan, 5th grade science.

Drawing meaningful conclusions from comparisons of adults’ and students’ experiences with blended learning is difficult, researchers say. College students, and adults taking on-the-job training courses, may have considerably more focus and motivation to use self-paced, computer-based instruction than K-12 students do, particularly at earlier grades.

One ongoing study by the RAND Corp. for the Gates Foundation is trying to tease out how blended learning plays out in lower grades. Researchers are chronicling how nearly 60 charter and regular district schools implement blended learning, and whether it improves student achievement.

An interim report, covering 23 charter schools and nearly 5,000 students, found students at two-thirds of schools that used blended learning made statistically significant gains on either math or reading tests. But researchers also warned that it isn’t yet possible to know “which particular instructional approaches may account for the positive student-learning outcomes.”

Of the schools that have seen gains, “it’s hard to say if these are just really strong charter schools, or if [blended learning] is playing a role,” said John F. Pane, a senior scientist at the RAND Corp., who is working on the evaluation.

Racing the Clock

The time frame for experimental research—five to seven years for most randomized controlled trials—is seen as a problem for conducting studies of education. It poses even greater obstacles for evaluating blended learning, in which iterations of software and curriculum can shift dramatically from year to year.

“The things we have a lot of evidence for—things like early-reading development, personalizing—took decades” to research, Ms. Mohammed said. “I’m not sure that sort of framework is going to be useful in blended learning, where the technology and innovations are changing so rapidly.”

From a research standpoint, online instruction would seem to have one clear advantage over in-person instruction: A researcher can’t follow a student’s thought process as he takes a test, but online programs like Carnegie’s Cognitive Tutor can track practically every keystroke entered by every student using the program, allowing analysis not only of what questions a student answers correctly, but how he approaches and works through a problem.

So-called “big data” hold potential to reveal whether a student really understands and can apply a concept, Mr. Pane said, but it’s proved to be far from a panacea. The positive results found in a recent evaluation of Cognitive Tutor’s blended-learning program, for example, remain in “a black box,” with researchers still unsure what exactly caused the improvements in math performance.

“It’s a huge volume of data and making sense of it is very challenging,” Mr. Pane said. “You get a bunch of keystrokes, but unless you know what was on the screen when those keystrokes were happening, it’s mind-bogglingly hard to analyze.”

Mr. Horn and Ms. Patrick call for less focus on evaluating individual blended-learning software and programs based only on test scores, and more support for identifying which outcomes schools want, then building measures to test those.

School leaders should not think of online or blended learning as fundamentally different from traditional classroom learning, Ms. Mohammed said. The research suggests they should instead implement it when they “want to solve specific instructional problems,” she observed.

“We have not focused on whether learning is actually different in the two different environments,” Ms. Mohammed said. “If you take an ineffective practice in face-to-face instruction and move it to an online setting, you shouldn’t expect better learning.”