At least since the first standards-and-testing regime began in K-12 public schools, advocates of all stripes have quarreled over the impact of the tests on teachers’ classroom instruction day to day, and whether the pressure to keep scores up warps teaching in unproductive ways. Let’s call it the Great Test-Prep War.
In this war, there are two major schools of thought. The first argues that test-prep strategies take valuable learning time away, because they focus on the rote or procedural knowledge that is most likely to appear on the tests, rather than conceptual learning. At its worst, it’s a form of cheating. (Ditch the tests!, they argue.)
The pro-accountability folks argue that it’s important to make sure schools are teaching kids something—and that as assessments grow more sophisticated and measure more complex skills, helping kids prepare for them will simply mirror good teaching. (Improve the tests!, they argue.)
So who’s right?
Actually, both sides seem to get it at least partly wrong, concludes a new study in Educational Researcher that offers a nuanced take at a controversial topic in K-12 education.
In an analysis of hundreds of videotaped teacher lessons, a pair of researchers found that explicit test-prep strategies were associated with declines in instructional quality—but not consistently across several school districts, and generally, not severely. So merely getting rid of standardized tests is unlikely to significantly elevate the quality of teachers’ lessons without other reforms, they say.
But the study throws cold water on the idea that better tests will lead to more consistent, higher-level instruction in classrooms. (That’s been a common refrain of the pro-testing folks.) In fact, teachers working in districts with more sophisticated testing regimes also taught weaker lessons when they engaged in test prep.
And although the researchers don’t go this direction in their paper, looking at these findings makes me ponder a reading-between-the-lines hypothesis: What if a lot of teaching just isn’t that ambitious to start off with? Is that why there’s not all that much of a difference between test-prep and regular lessons?
The study, by David Blazar of the University of Maryland College Park, and Cynthia Pollard of the Harvard Graduate School of Education, looks at the instruction of about 300 4th and 5th grade teachers. The teachers were surveyed about how often they used various test-preparation strategies in class, and each of them submitted as least three and as many as nine or ten lessons for analysis. (A subset of the 60 teachers submitted lessons that were both explicitly focused on test prep and regular lessons that didn’t feature it.)
The teachers worked in five districts—two that used a standardized math test that had more cognitively demanding tasks and open-ended test questions, and three others whose test measured basic skills mainly via multiple-choice questions.
Then, the lessons were analyzed against a part of a math teaching framework called Ambitious Mathematics Instruction. It measures whether teachers engaged in complex instruction—such as whether teachers offered different ways of mathematical modeling, offered explanations, and helped to draw out and clarify students’ math knowledge.
In their analysis, the researchers controlled for characteristics like the teachers’ prior training and experience and other factors that might have skewed the analysis.
Test Prep: Not As Bad As They Say, Not As Good As They Say.
Here’s one important finding: Test preparation is common. In the districts studied, teachers reported that they engaged in various forms—such as using test items as part of instruction or reallocating time to cover topics likely to appear on the test—between 1-2 and 3-4 times a week. The precise amounts did differ among the five districts.
And when comparing lessons from the same teachers that explicitly engaged in test prep with those that didn’t, the researchers found that test preparation was associated with a quarter of a standard deviation decline in complex math teaching.
“The narrative for years is that this is horrible for students, and we might have expected to see really, really large effects, and that’s not the case,” said Blazar. “They were engaging in test prep a few days a week, and they were doing it fairly consistently. If it was so horrible we might expect to see larger decreases or detriments, and these point estimates are meaningful, but not huge.”
Nor were the results consistent from the five districts the researchers studied: They showed up most strongly in one of the districts that used a more-complex math test.
One possibility, the authors said, is that in that district, instruction was generally higher and more ambitious. So when teachers there engaged in test-prep, it looked markedly different from typical instruction. In other districts, there might have been less of a difference between regular and test-prep-oriented lessons.
On the other hand, the finding also appears to disprove the idea, pervasive among some so-called education reformers, that a better test will ameliorate the negative effects of teaching to the test.
To sum up, getting rid of tests is unlikely to solve the ills of uneven instructional quality.
“It is not a silver bullet,” Blazar said. “It very well could be, though, that some comprehensive efforts that look at assessments and how they’re aligned to standards and professional development and other resources may be more of a driver to elevate quality of instruction.”
A version of this news article first appeared in the Curriculum Matters blog.