Assessment

Does Test Prep Harm Teaching? Maybe Not as Much as We Think

By Stephen Sawchuk — October 13, 2017 4 min read
  • Save to favorites
  • Print

At least since the first standards-and-testing regime began in K-12 public schools, advocates of all stripes have quarreled over the impact of the tests on teachers’ classroom instruction day to day, and whether the pressure to keep scores up warps teaching in unproductive ways. Let’s call it the Great Test-Prep War.

In this war, there are two major schools of thought. The first argues that test-prep strategies take valuable learning time away, because they focus on the rote or procedural knowledge that is most likely to appear on the tests, rather than conceptual learning. At its worst, it’s a form of cheating. (Ditch the tests!, they argue.)

The pro-accountability folks argue that it’s important to make sure schools are teaching kids something—and that as assessments grow more sophisticated and measure more complex skills, helping kids prepare for them will simply mirror good teaching. (Improve the tests!, they argue.)

So who’s right?

Actually, both sides seem to get it at least partly wrong, concludes a new study in Educational Researcher that offers a nuanced take at a controversial topic in K-12 education.

In an analysis of hundreds of videotaped teacher lessons, a pair of researchers found that explicit test-prep strategies were associated with declines in instructional quality—but not consistently across several school districts, and generally, not severely. So merely getting rid of standardized tests is unlikely to significantly elevate the quality of teachers’ lessons without other reforms, they say.

But the study throws cold water on the idea that better tests will lead to more consistent, higher-level instruction in classrooms. (That’s been a common refrain of the pro-testing folks.) In fact, teachers working in districts with more sophisticated testing regimes also taught weaker lessons when they engaged in test prep.

And although the researchers don’t go this direction in their paper, looking at these findings makes me ponder a reading-between-the-lines hypothesis: What if a lot of teaching just isn’t that ambitious to start off with? Is that why there’s not all that much of a difference between test-prep and regular lessons?

Studying Lessons

The study, by David Blazar of the University of Maryland College Park, and Cynthia Pollard of the Harvard Graduate School of Education, looks at the instruction of about 300 4th and 5th grade teachers. The teachers were surveyed about how often they used various test-preparation strategies in class, and each of them submitted as least three and as many as nine or ten lessons for analysis. (A subset of the 60 teachers submitted lessons that were both explicitly focused on test prep and regular lessons that didn’t feature it.)

The teachers worked in five districts—two that used a standardized math test that had more cognitively demanding tasks and open-ended test questions, and three others whose test measured basic skills mainly via multiple-choice questions.

Then, the lessons were analyzed against a part of a math teaching framework called Ambitious Mathematics Instruction. It measures whether teachers engaged in complex instruction—such as whether teachers offered different ways of mathematical modeling, offered explanations, and helped to draw out and clarify students’ math knowledge.

In their analysis, the researchers controlled for characteristics like the teachers’ prior training and experience and other factors that might have skewed the analysis.

Test Prep: Not As Bad As They Say, Not As Good As They Say.

Here’s one important finding: Test preparation is common. In the districts studied, teachers reported that they engaged in various forms—such as using test items as part of instruction or reallocating time to cover topics likely to appear on the test—between 1-2 and 3-4 times a week. The precise amounts did differ among the five districts.

And when comparing lessons from the same teachers that explicitly engaged in test prep with those that didn’t, the researchers found that test preparation was associated with a quarter of a standard deviation decline in complex math teaching.

“The narrative for years is that this is horrible for students, and we might have expected to see really, really large effects, and that’s not the case,” said Blazar. “They were engaging in test prep a few days a week, and they were doing it fairly consistently. If it was so horrible we might expect to see larger decreases or detriments, and these point estimates are meaningful, but not huge.”

Nor were the results consistent from the five districts the researchers studied: They showed up most strongly in one of the districts that used a more-complex math test.

One possibility, the authors said, is that in that district, instruction was generally higher and more ambitious. So when teachers there engaged in test-prep, it looked markedly different from typical instruction. In other districts, there might have been less of a difference between regular and test-prep-oriented lessons.

On the other hand, the finding also appears to disprove the idea, pervasive among some so-called education reformers, that a better test will ameliorate the negative effects of teaching to the test.

To sum up, getting rid of tests is unlikely to solve the ills of uneven instructional quality.

“It is not a silver bullet,” Blazar said. “It very well could be, though, that some comprehensive efforts that look at assessments and how they’re aligned to standards and professional development and other resources may be more of a driver to elevate quality of instruction.”

Image: Getty

Related Tags:

A version of this news article first appeared in the Curriculum Matters blog.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Spotlight Spotlight on Assessment in 2021
In this Spotlight, review newest assessment scores, see how districts will catch up with their supports for disabled students, plus more.
Assessment 'Nation's Report Card' Has a New Reading Framework, After a Drawn-Out Battle Over Equity
The new framework for the National Assessment of Educational Progress will guide development of the 2026 reading test.
10 min read
results 925693186 02
iStock/Getty
Assessment Opinion Q&A Collections: Assessment
Scores of educators share commentaries on the use of assessments in schools.
5 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Standardized Tests Could Be in Jeopardy in Wake of Biden Decisions, Experts Say
Has the Biden administration shored up statewide tests this year only to risk undermining long-term public backing for them?
6 min read
Image of a test sheet.
sengchoy/iStock/Getty