The disruptions in online assessments that roiled Montana, Nevada, and North Dakota in spring 2015 appear to have had both negative and positive effects on individual students’ test scores, finds an analysis.
While some students’ scores appeared to have been hurt by “logout” interruptions, evidence suggests others may have actually benefited from temporarily getting booted out of the system, a conclusion the authors base on statistical modeling. The effects varied greatly: Most districts in Montana and North Dakota had no students forced to cope with a logout interruption, but in some school systems in those states, all students ran into problems.
The independent analysis was conducted by the National Center for the Improvement of Educational Assessment, a research and consulting nonprofit hired by the Smarter Balanced Assessment Consortium, the group that sponsored the exams. The review was limited: Vendors’ testing systems only collected data on one type of interruption—students getting kicked out because of server failure.
