Big name standardized assessments—like the SAT, ACT, and many state exams—give test takers the opportunity to listen to a question read aloud before they answer it.
Does this feature improve student performance, or create a technological distraction?
Digital Promise, a nonprofit organization that works on equity and technology issues in schools, put the so-called “text-to-speech” component to the test, analyzing how 8th graders used it in a geometry problem on the National Assessment of Educational Progress.
The picture that emerged was “nuanced,” said Xin Wei, a senior quantitative researcher at Digital Promise, who led the study.
Overall, 7 percent of students, roughly 2,000 test takers out of about 26,000, took advantage of the text-to-speech feature. Those who used it were more likely to be English learners, students in special education, students of color, or students who received an accommodation for extended testing time.
The lowest-performing students were more likely to get the question correct if they used text-to-speech. In fact, those who listened longer tended to do better—to a point. The effect plateaued at 25 seconds of listening.
Using text-to-speech made no difference for the highest-performing students, those who scored at the “proficient” or “advanced” level on the test.
Students with mid-range performance on the test overall were less likely to get the question right if they used the text-to-speech option but repeatedly toggled it on and off.
That might mean the “tool was creating a distraction or interrupting the cognitive flow rather than supporting the comprehension,” Wei said.
“This study offers evidence [that] ed tech [can] actually support learning, but it depends on how it is used and by what group of students,” Wei said. For “the lowest-achieving student, text-to-speech is helping those who decide to use it and put in the effort to listen to the whole problem before they answer the questions.”
Students need to master text-to-speech technologies before test day
The findings also suggest teachers shouldn’t wait until the day of a standardized test to help students figure out how to use the text-to-speech feature, Wei said.
Teachers should treat text-to-speech tools the same they would “any problem-solving strategy,” Wei said. Students “need to practice how to use it. The teacher [needs] to model good use of it.”
The feature—which is integrated into many digital learning tools, not just those used for assessments—should be a part of “everyday instruction” well before testing day, Wei added.
What’s more, the study offers a window into how ed-tech tools can be helpful—and where they may cause more harm than good, Wei said.
That’s meaningful at a time when states are taking steps to limit screen time in schools, and education technology in general faces significant backlash from some educators and parents.
Instead of asking the question whether ed tech in general works, educators need to consider whether a particular tool can work for a certain group of students, doing a specific task, she said.
“Every tool needs to be studied carefully, and then we can figure out how students can best use it,” Wei said.
