Teachers are scientists, they’ve always experimented. Most of the time it’s informal, “Let’s try a new behavior management routine.” Or, “Watch this video
tonight and we’ll discuss it in the morning.” Or, “Let’s try a really hard problem.”
It’s been hard to run formal experiments because measurement systems in education are so bad, a year end test won’t help with a reading experiment this
week.
What if we gave teachers, and whole schools, the chance to run experiments of their own or to join larger trials? What if they had access to better
measures and powerful analytics?
This sort of experimentation is common in business where a number of big vendors (SAS, SAP, Microsoft, IBM) provide business analytics to support quick
experiments. For example, APT, acquired earlier this year by MasterCard has, “cloud-based software
that determines cause-and-effect relationships between tested programs and key performance metrics. Beyond determining the overall impact of each
initiative, by rapidly sifting through huge data-sets, the software helps companies target their actions to the customers predicted to respond most
profitably.”
In software development it’s common to run A/B tests that quickly test user experience (UX)
features. Rapid feedback is critical to iterative development.
The combination of social media and quantified self is ushering an age of health
experimentation. There are advocates for self-experimenting, trials with an n-of-one.
Ian Eslick, who went back to MIT for a mid-career PhD, has studied self-experimentation. He’s interested in efficacy not validity. He’s willing to be wrong
for a while but wants to make quick practical progress. He wants to help people find treatments that work for them, fast. “As individuals we make the best
progress when our hypotheses are well founded and the conclusions drawn from our experiments are accurate. As a community we make the best progress when we
can share our ideas in ways that accurately predict how someone else will fare.”
We need more of this in education, and it’s getting easier to manage. Powerful and easy to use formative assessment systems (from Edmodo, MasteryConnect, and OpenEd, for example) make it easy for a school,
district, or network to deploy common assessments around an quick A/B test.
Intelligence platforms like BrightBytes improve
hypothesis generation, as well as measurement of results over time. In business, this is typically thought of as Return on Investment, or ROI. BrightBytes
thinks of it as ROL, Return on Learning.
Test what? If teachers and schools had the ability to run short trials, what experiments would they run? Some examples include comparing:
-
Reading strategies and materials
-
Blended learning models (e.g., lab rotation vs station rotation)
-
Math software
-
Behavior management systems
Like projects, experiments can be assessed by a mixture of quantitative and qualitative factors. A variety of measurement systems can be used including
formative assessment, teacher and student surveys, and observations.
Why not just rely on national studies? Results in education are highly context specific, particularly implementation efficacy. As a result, national
studies may be suggestive but they won’t tell you how something will work at your school. And they’re not available for a new app or strategy.
The first head of the Institute of Education Sciences, Russ Whitehurst set a gold standard for research
particularly around the use of randomized controlled trials (RCT). There is obvious benefit to setting a high bar for evidence but we often need better
evidence faster and cheaper than is possible through RCT.
I’ve complained before that many randomized trials (like this 2013 small school study)
providing a blinding flash of the obvious a decade late. They sacrificed efficacy for validity.
We need well designed large scale controlled studies but there is a growing need to support an iterative change strategy in education.
Path forward. I’m optimistic about the shift to personalized learning not only for the customization and motivation students will experience, but for the
tools that it will provide to teachers. Big Data will allow interested teachers to become data scientists and participate in large scale experiments in
real time.
With support from the Gates Foundation, organization
like LEAP Innovations are creating test beds of groups of schools hosting short cycle trials.
This is a big step in the right direction.
The next step in supporting teachers as scientists is an education intelligence platform that combines information about best practices, easy to use
formative assessment, templates for setting up experiments, and social features for sharing.
For more, check out: