I’ll bet most school administrators wish they had a nickel for every peddler who comes around with an educational program or product that is purportedly research-based. The truth is that “research-based” is a label that means different things to different people.
In an effort to develop a more common understanding of what it takes for a product to be truly research-based, the Software & Information Industry Association, with the help of prominent education researchers, has developed guidelines for best practices in evaluation research.
The association is the primary trade group for providers and developers of software and digital content, and its members include big names such as Blackboard Inc., Houghton Mifflin, and McGraw-Hill. The new guidelines are aimed at mostly at the managers within those organizations who might be tasked with planning, designing, conducting, and reporting evaluation studies of new products and services.
The standards don’t specify or endorse any particular research methodology. After all, this group isn’t the What Works Clearinghouse. These folks don’t want to see their products yield no effects.
But the guidelines do offer sensible principles for evaluation research. They emphasize the importance of choosing the right outcome measure, ensuring that studies are large enough or last long enough for effects to register, and accurately translating results into reports that customers can easily access, among other principles. Educational administrators who have to select new technology products for the classroom might find these guidelines useful, too. Are your vendors adhering to them?