Data Opinion

Does Studying Student Data Really Raise Test Scores?

Researchers say the popular practice is largely ineffective
By Heather C. Hill — February 07, 2020 5 min read
  • Save to favorites
  • Print

Question: What activity is done by most teachers in the United States, but has almost no evidence of effectiveness in raising student test scores?

Answer: Analyzing student assessment data.

This practice arose from a simple logic: To improve student outcomes, teachers should study students’ prior test performance, learn what students struggle with, and then adjust the curriculum or offer students remediation where necessary. By addressing the weaknesses revealed by the test results, overall student achievement would improve.

My own recent experiences visiting schools imply this trend continues."

Yet understanding students’ weaknesses is only useful if it changes practice. And, to date, evidence suggests that it does not change practice—or student outcomes. Focusing on the problem has likely distracted us from focusing on the solution.

With the birth of large-scale state assessments and widening data availability in the 1990s, school leaders and teachers could access information on student performance that was common across schools and classrooms. Many schools also instituted standardized “interim” assessments, claiming that this periodic, low-stakes testing could help teachers identify difficult content and struggling students before the state assessment, giving both teachers and students a chance to catch up. Over time, educational testing and data companies including the Achievement Network, NWEA, and McGraw-Hill’s Acuity began to sell interim assessments to schools and states, making such assessments (and their cousins, test-item banks that support formative assessment) a billion-dollar business.

Currently, a large number of teachers report they regularly get together to analyze student assessment results. In a 2016 survey by Harvard’s Center for Education Policy Research, 94 percent of a nationally representative sample of middle school math teachers reported that they analyzed student performance on tests in the prior year, and 15 percent said they spent over 40 hours that year engaged in this activity. Case-study research suggests that in many Title 1 schools, this activity is a cornerstone of teachers’ weekly or monthly collaborative time.

About this series


This essay is the second in a series that aims to put the pieces of research together so that education decisionmakers can evaluate which policies and practices to implement.

The conveners of this project—Susanna Loeb, the director of Brown University’s Annenberg Institute for School Reform, and Harvard education professor Heather Hill—have received grant support from the Annenberg Institute for this series.

Read the full series here.

But here’s the rub: Rigorous empirical research doesn’t support this practice. In the past two decades, researchers have tested 10 different data-study programs in hundreds of schools for impacts on student outcomes in math, English/language arts, and sometimes science. Of 23 student outcomes examined by these studies, only three were statistically significant. Of these three, two were positive, and one negative. In the other 20 cases, analyses suggest no beneficial impacts on students. Thus, on average, the practice seems not to improve student performance.

One critical question, of course, is, why?

Observational studies suggest that teachers do, in fact, use interim assessments to pick out content that they need to return to. For instance, in a study published in 2009, Margaret E. Goertz and colleagues at the University of Pennsylvania observed teachers planning to revisit math topics using a combination of whole-group and small-group instruction.

But Goertz and colleagues also observed that rather than dig into student misunderstandings, teachers often proposed non-mathematical reasons for students’ failure, then moved on. In other words, the teachers mostly didn’t seem to use student test-score data to deepen their understanding of how students learn, to think about what drives student misconceptions, or to modify instructional techniques.

My own recent experiences visiting schools imply this trend continues. Field notes from teacher data-team meetings suggest a heavy focus on “watch list” students—those predicted to barely pass or to fail the annual state reading assessment. Teachers reported on each student, celebrating learning gains or giving reasons for poor performance—a bad week at home, students’ failure to study, or poor test-taking skills. Occasionally, other teachers chimed in with advice about how to help a student over a reading trouble spot—for instance, helping students develop reading fluency by breaking down words or sorting words by long or short vowel sounds. But this focus on instruction proved fleeting, more about suggesting short-term tasks or activities than improving instruction as a whole.

Common goals for improving reading instruction, such as how to ask more complex questions or encourage students to use more evidence in their explanations, did not surface in these meetings. Rather, teachers focused on students’ progress or lack of it. That could result in extra attention for a watch-list student, to the individual student’s benefit, but it was unlikely to improve instruction or boost learning for the class as a whole.

In reviewing the research on teachers analyzing student data, I came across a small number of programs that included interim assessment as one part of a larger instructional package. While I excluded these studies from the formal review I undertook for this essay, they are notable nonetheless. One, by Janke M. Faber and colleagues in the Netherlands, focused on a program that not only contained computer-based interim assessments but also provided both instructionally focused feedback to teachers and students and personalized online student assignments. Another study, led by Jonathan A. Supovitz and colleagues at the University of Pennsylvania, examined the Ongoing Assessment Project, a program that helps teachers create assessments and examine the results, then combines this practice with professional development focused on mathematics content and student thinking about that content. Both of these studies saw positive impacts, suggesting that the analysis of data can, when combined with strong supports for improved teaching, shift student outcomes. But the small number of programs that combine the study of data with wider instructional supports limits our ability to draw real conclusions.

In total, the research in this area suggests that district and school leaders should rethink their use of state and interim assessments as the focus of teacher collaboration. Administrators may still benefit from analyzing student assessment results to know where to strengthen the curriculum or to provide teacher professional learning. But the fact remains that having teachers themselves examine test-score data has yet to be proven productive, even after many trials of such programs.

For many schools, this news is disheartening. Retooling teacher collaborative time will be a major shift—and that’s assuming that schools can first identify more effective ways to help teachers improve their instruction. In our next column, we’ll cover possible replacement activities.

Editor’s Note: This is part of a continuing series on the practical takeaways from research.
A version of this article appeared in the February 12, 2020 edition of Education Week as Does Studying Student Data Really Raise Test Scores?


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Pave the Path to Excellence in Math
Empower your students' math journey with Sue O'Connell, author of “Math in Practice” and “Navigating Numeracy.”
Content provided by hand2mind
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Recruitment & Retention Webinar
Combatting Teacher Shortages: Strategies for Classroom Balance and Learning Success
Learn from leaders in education as they share insights and strategies to support teachers and students.
Content provided by DreamBox Learning
Classroom Technology K-12 Essentials Forum Reading Instruction and AI: New Strategies for the Big Education Challenges of Our Time
Join the conversation as experts in the field explore these instructional pain points and offer game-changing guidance for K-12 leaders and educators.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Data Whitepaper
MTSS Resources to Support Data Analysis Teams This School Year
Share these tools with your team to help them more confidently navigate essential processes, such as identifying learners’ strengths and ...
Content provided by Renaissance
Data 'Hidden Homeless': A Key Measure of Homelessness Excludes Most Students
Federal agencies differ in how they measure homelessness—and many vulnerable students are left out.
3 min read
Photograph of a low angle view of children with backpacks climbing the school staircase.
Data Spotlight Spotlight on Leveraging Data for Student Success
This Spotlight will help you learn how data can help schools target resources, explore how to improve instruction with data, and more.
Data What Superintendents Say They Need More of to Help Them Manage Districts
98% of those surveyed said better data would make them more comfortable making decisions.
2 min read
Image of a data dashboard.
Suppachok Nuthep/iStock/Getty