At the Waukesha STEM Academy-Saratoga Campus, students can find themselves taking adaptive tests once a day, or once a month, depending on the subject matter and how quickly they work through the curriculum. But the Wisconsin charter school found the customized assessments that adjust the level of difficulty based on the responses of the test-taker so beneficial that school leaders siphoned funds away from a full-time educational position to buy more of this personalized testing.
Students have performed well on the adaptive tests, which appears to have translated to state testing. Less than 5 percent scored in the 24th percentile or lower on state tests, compared with nearly 20 percent five years ago when the school was not using adaptive testing tools.
But it was an inadvertent benefit of adaptive testing that demonstrated to educators at the 336-student school just how individualized their instructional approach could be, whether in the classroom or during pullout interventions.
Because each student tends to use this type of testing differently, the school has students fill out “learner profiles” within the first month of each school year to help teachers best use adaptive-assessment software like Assessment and Learning in Knowledge Spaces, or ALEKS, which measures an understanding of math concepts, and Achieve 3000, which supports reading comprehension and writing proficiency. The profiles ask about preferred learning methods (videos, interactive activities, direct instruction) and learning environments (collaborative, one-on-one, noise-level impact), and solicit opinions on instructional techniques the school’s 6th, 7th, and 8th graders believe are least and most effective.
“We were amazed when we realized we could dive deeper into every student as an individual learner and not just think about where they should be, based on their age,” said Principal Jim Murray. “We were able to reformulate how we were approaching kids.”
Both the results of the adaptive tests, and the student profiles, are used to shape instruction at Waukesha STEM Academy. Students discuss their performance with their teachers, who analyze it with a data coach before results are passed on to parents. If a student’s scores on adaptive tests differ significantly from scores on regular coursework, another form of assessment—a preference taken from the learner profile—can be used to tailor instruction to their specific needs.
Lessons Learned
Computer-adaptive testing continues to become more prevalent and more sophisticated. It is being used by 22 states in the Smarter Balanced Assessment Consortium, one of two main groups of states crafting tests aligned to the Common Core State Standards. Meanwhile, algorithms for classroom-based adaptive-testing software have evolved to track not just academic proficiency but metacognitive skills such as confidence and the ability to reflect.
A growing number of districts around the country are integrating adaptive testing into their daily routines. As a result, administrators and teachers have had time to learn important lessons about what works—and what doesn’t—when it comes to using technology-based assessments to tailor instruction.
Milissa Crum, a general education teacher at the 1,500-student Highland Middle School in Anderson, Ind., uses iLit, an adaptive literacy program—which embeds adaptive testing into the curriculum—every day with 51 6th graders, nearly half of whom have individualized education programs and all of whom are at least two grades below level in reading.
The program helped her realize that the goals she was encouraging students to set, such as earning an A, were too general. She now has students keep intricate data from their personalized adaptive assessments in binders, with specific objectives—such as reading 100,000 words every month, or improving reading-level scores by a certain percentage.
And there have been other teachable moments through iLit: “I need to shut my mouth some of the time,” Ms. Crum acknowledged. “So much of what students are expected to do in this program is about reflecting and responding, and our conversations are so much richer now because of the opportunities they have to share their ideas and let their voices be heard.”
SuccessMaker, a reading and math curriculum that integrates adaptive testing, is used with special education students in the 42,000-student Killeen Independent School District in Texas. School officials say the curriculum has fostered a sense of academic ownership in a school system with a student-mobility rate of nearly 30 percent, movement attributable to the district’s proximity to the Fort Hood military base. Students use the program four times a week, 30 minutes each day for reading and 15 minutes for math.
The disruption in education that comes with frequent military transfers is especially formidable for the district’s 4,700 students with special needs, according to Lynn Young, the executive director for special education. But adaptive testing has succeeded with the population where previous efforts have fallen short, she said.
Because the program gathers baseline data, a step Ms. Young said the district hadn’t previously considered, schools can correlate their adaptive testing results with state standards and offer quicker interventions.
“We can now zero in on why a student may not be getting a concept because earlier concepts are lacking, [and] then increase practice in areas where extra support is needed,” she said. Students who may need to review the same concept 50 times to understand it can now do so at their own pace without getting frustrated, she added. One 8th grader, Ms. Young noted, credited the program with helping him pass the Texas Essential Knowledge and Skills assessment—the first time he’d passed a state assessment since his first attempt in 3rd grade.
Test-Taking Accommodations
At the 570-student Bunker Hill Elementary School in Middletown, Del., some students are permitted to take adaptive state assessments over two sessions instead of one, noting that they feel less stressed and more confident when there is time to pace themselves and check over their work.
“It’s good to take breaks,” said Evan Bush, a 5th grader. “The next time you come back [to the test], you know exactly what to do. You’re more focused because you’ve got the routine down.”
Adapting to the tests has been essential, pointed out Principal Laurie Wicks. “We have some children who have focus issues, and you can see halfway through the test that all they do is click, click, click, so we typically give the tests over a couple of sessions,” she said. On the other hand, “a high percentage of our really top kids want to sit there and plug and plug. We’ve found if you split the tests for them, they don’t do as well because they have that drive and don’t want to stop. You have to know your students and what works best for them.”
Adaptive assessments are also driving instructional changes at Bunker Hill. Teachers make it a point to show the youngest students, who tend to wrestle with questions that measure mastery of content in multiple ways, a variety of assessment prompts and vocabulary words before test time to minimize confusion and frustration. And teachers have been able to gain a deeper understanding of the ability levels of students who score extremely high on adaptive assessments.
Still, Ms. Wicks would like to see adaptive assessments improve to the point where they are dispensing more useful information to educators. She wants to know not only whether students have difficulty with math when it comes to geometric reasoning, for example, but also the type of question and which concept within that strand causes the trouble.
“The data that I want and that I think is beneficial needs to be drilled down to tell me specifically what a child can and can’t do,” she said, instead of validating with documentation what teachers often already know. “What we get is too general. ... This is definitely an area that needs refinement in the future.”
