As the roll out of the assessments for the Common Core State Standards approaches, school districts have been spending millions of dollars per year on diagnostic exams in the hopes that these interim results will help improve scores on high-stakes state exams given toward the end of the school year.
But research presented this week at the annual meeting of the American Educational Research Association in Philadelphia suggests that, though diagnostic assessments may lead to some increases in 3rd-8th grade math scores, they have no effect in reading and a small negative effect in the lower grades in both subjects.
The paper, written by researchers at Michigan State University and the American Institutes for Research and funded by the federal Institute of Education Sciences, summarized the results of two different experiments that took place in Indiana. In 2009-10, 20,000 students in 50 schools participated in the research. Half were randomly assigned to take the popular mCLASS and/or Acuity interim assessments throughout the school year. Treatment group students in grades K-2 took mClass. Treatment group students in grades 3-8 took Acuity. The remaining students were the control group, so they went about business as usual.
In 2010-11, a separate but similar experiment included a total of 30,000 students in 50 schools.
In math, the interim exams had a small but positive effect on 3rd-8th graders’ scores on Indiana’s summative statewide exams, which are called the Indiana Statewide Testing for Educational Progress-Plus or ISTEP. For a typical student, the effect was equivalent of moving from the 50th to the 53rd percentile in math.
However, the researchers found no significant effects on 3rd-8th graders’ reading ISTEP results.
And, perhaps most surprisingly, they found a small negative effect on the reading and math results of kindergarten-2nd graders who took mClass. The effect was the equivalent of moving from the 50th to the 44th percentile in math and from the 50th to the 45th percentile in reading.
A caveat is that kindergarten-2nd grade students do not take the ISTEP. So researchers had to administer the Terra Nova math and reading exams instead.
“One possibility is that the outcome (Terra Nova scores) does not capture well the intention of the treatment,” wrote Michigan State associate professor Spyros Konstantopoulos and his co-authors. “Perhaps interim assessments are more effective in mathematics and in later grades, especially grades 5 and 6.”
Acuity is a 30-to-35-question online reading and math assessment that is aligned with ISTEP. Both tests are created by CTB/McGraw-Hill. Teachers can construct additional tests or assign pre-packaged exercises, based on the test results.
Indiana teachers conduct mCLASS reading and language arts assessments in person, with help from a personal digital assistant, while mClass math assessments are pen-and-paper exams. The teacher enters the data in the computer, which spits out results.
Teachers were supposed to receive training on both types of diagnostic exams before using them. All schools statewide were scheduled to start using the diagnostic assessment this school year.
A version of this news article first appeared in the Inside School Research blog.