No responsible investor would invest in a health-care-technology product that wasn’t thoroughly researched by top medical scientists. No one would invest in energy technology whose potential was unsupported by the studies and practices of relevant experts. Yet when it comes to education technology, the logical connection between evidence of effectiveness and the wisdom of investment decisions is often ignored.
It’s not for lack of relevant research. Throughout my tenure as president of Teachers College, I have marveled at how disconnected the instructional technology products have been from the groundbreaking work of our faculty and students (and of researchers elsewhere) in the learning sciences, classroom use of technology, learning analytics, and evaluation. The work of other major research universities is similarly overlooked by the ed-tech industry. Investors in other sectors, such as health and energy, eagerly draw on a wealth of researcher consultants and even firms to facilitate the connection between evidence and investment decisions. In education, however, aside from some big companies that can afford substantial studies, I have noted a hesitancy to invest in research on effectiveness, particularly among startup developers and venture capitalists.
I’ve heard many reasons for this hesitancy: an insufficient rate of investment opportunity in ed tech; a dysfunctional market, where educator purchasers value user testimonies and personal connections over evidence of effectiveness; and concerns that sharing data with researchers will expand the circle of those who might endanger student privacy. I also infer some deeper issues, including the age-old sentiment that anyone who went to school knows what works, as well as a distrust of researcher jargon and studies that appear inaccessible.
I also suspect a lack of appreciation within the ed-tech industry for how much investment good research requires. Educational research encompasses everything from studies that inform initial design to formal evaluation once a product has already been in use for a sufficient period of time. Research includes a variety of methodologies, including keystroke data analysis, classroom observation, and student outcome comparisons. While relatively inexpensive market research can produce data about the extent of usage or page views, which may be helpful in sales, those data do not tell us whether educational products are working. Research that aims at producing evidence of effectiveness is likely to be much more expensive.
But as is true in other industries, I believe that investing in better research will lead to increased market share and economic value over time. The throw-it-against-the-wall mentality—the notion that if one thing fails, something else will quickly take its place—is an inefficient alternative to research-based investment. Worse, it’s an upsetting failure of responsibility to students, who go through school only once. They don’t have time to keep trying bad products. Nor should scarce taxpayer money be gambled on products that will end up being a waste of time.
All this leads me to proclaim a set of resolutions for the ed-tech industry:
1. Use what we know about learning to develop product designs. Instead of building instructional products around an idea of learning, seek research and advice about how best to teach a particular topic, order concepts, present challenging tasks and assess learner knowledge. (The ABCs of How We Learn: 26 Scientifically Proven Approaches, How They Work, and When to Use Them by Daniel L. Schwartz, Jessica M. Tsang, and Kristen P. Blair is a great primer on the subject.)
The throw-it-against-the-wall mentality is an inefficient alternative to research-based investment."
2. Put educators or researchers on advisory boards of startups. Right now, investors serve on advisory boards, providing much-needed business-organization, marketing, sales, and financial expertise. Educational expertise—about learning and about how products work in classrooms and with various curricula—is just as critical for developing new ed-tech products.
3. Try products out in real settings, revising them again and again until they show promise for delivering results. Ideally, involve researchers in this task so trials provide valid and usable information.
4. Invest in teacher and school-leader professional development. School-based products are most effective when teachers know how to use them well, how to fit them into lessons or use them to supplement teacher-designed lessons, and how to guide and assess productive student use. Similarly, leaders should support teacher technology use and know how to determine whether teachers are using products effectively.
5. View implementation as an essential part of the process. Does the hardware work? How do the schools access the product most easily? Are data useful and usefully displayed? Include technical assistance, or coordination with in-school technology specialists, as part of the price package.
6. Work with buyers to determine how they judge effectiveness. Are they focusing on standardized tests as sole measures, or are they interested in other measures of effectiveness, such as teacher reports and student engagement? Make sure products can produce evidence on the measures valued by users.
7. Safeguard student privacy. The data collected through software use are intimate data about student learning that provide important insights about how students think. Therefore, security issues and protecting student identities are very important to educators and parents. Many school districts have developed contract language to assure such protections; both sellers and buyers should learn from one another about effective approaches. Researchers, too, must attend to privacy concerns; in the past, the field has not educated itself about adequate security.
8. Document when evidence about effectiveness helps sales and increases market share. Over time, research about the value of research may be the most important evidence we can gather to make the case for a focus on effectiveness.
I realize these are ambitious resolutions. But given the urgency of improving education in our country, especially in this pivotal moment, we can’t afford to settle for less.
