Data Opinion

Standardized Tests Aren’t the Only Meaningful Data on Student Achievement

The case for using “street data”
By Shane Safir — September 19, 2022 5 min read
conceptual image of a young person walking a trendline
  • Save to favorites
  • Print

Public education needs a new data paradigm. The past 20 years have trapped us in a narrow definition of data that peddles a deficit narrative about Black, Indigenous, Latinx students, students with learning needs, and other historically marginalized groups. This era has stripped educators of agency and made student voice peripheral to our accounting of “school progress,” which we largely measure through incremental gains in test scores and other quantitative metrics like attendance and graduation rates.

It’s time to reimagine “data” for the next generation of schooling, beginning with street-level qualitative data that is rooted in student experience and invites educators to become ethnographers rather than statisticians in their pursuit of educational equity.

After this month’s release of data from the National Assessment of Educational Progress showed significant dips in student math and reading scores, national coverage of the results has told a story of “learning loss” and the need to fill student “gaps.” This framing echoes nearly two decades of test-centered solutions that fail to account for the complexity of learning.

Complex times call for complex approaches, including an orientation toward a love of learning rather than the deficit-based loss: How can we help children—and teachers—cultivate joy in the wake of so much personal loss and tragedy? We must embrace an expansive concept of data, one that centers student and family voices as well as students’ cultural values, funds of knowledge, identity development, sense of belonging, and mastery of 21st century competencies that extend well beyond test-taking.

1. We need to adjust the role of big, quantitative data in school improvement efforts. The entire premise that we can define “learning loss” as a function of how students should have scored on standardized tests is problematic, dangerous, and dehumanizing. Achievement is about so much more than test scores, and authentic assessments of achievement would incorporate student voice, reflection, and performance on real-world learning tasks, not just tests.

When we only evaluate learning through such analysis as “the average high-poverty school that remained in remote instruction for a majority of 2020-21 lost roughly .44 standard deviations in achievement” (from the recent report out of Harvard University, “The Consequences of Remote and Hybrid Instruction During the Pandemic”), we pursue a path of uncreative solutions that will further deepen opportunity gaps and reproduce status quo approaches.

Author and leadership coach Jamila Dugan calls this overemphasis of large-scale, quantitative data the “boomerang equity trap": investing time and resources to understand your equity challenges but reverting back to previous mental models in ways that lead to unintentionally harmful solutions.

To suggest—as the Harvard University report does—that interventions like high-dosage tutoring or remediation will produce incremental gains (i.e., a .38 standard deviation gain in math) exhibits an incarceration of the imagination that simply doesn’t speak to parents and students. Ask any parent of school-age children what matters most to them in the wake of a traumatic global pandemic, and I guarantee they will not clamor for solutions measured by a .38 standard deviation gain in math.

This type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just "unplugged" for a while and lost valuable "processing" time.

This is not to say that increasing numeracy and literacy doesn’t matter; of course, it does. However, this type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just “unplugged” for a while and lost valuable “processing” time, the solution to which is to simply run them overtime with high-dosage tutoring to get them where they need to be.

2. We need to shift our focus from so-called “achievement gaps” to opportunity gaps by collecting qualitative information that is close to the learner. This qualitative information is what I call “street data.” Instead of fixating on filling purported “gaps” in learning, street data can allow education researchers to start addressing opportunity gaps, or what Gloria Ladson-Billings refers to as the lingering “education debt” owed to Black, Indigenous, and other historically marginalized groups of students.

This would mean building more robust and holistic data systems that highlight student and community voice—from school planning templates to report cards to district equity audits that don’t just include numbers but tell a story of change. Researchers should particularly focus on learning more from school districts that have succeeded in closing gaps and developing innovative approaches to supporting underresourced schools.

In contrast to quantitative data, street data offers a wider lens to start measuring what truly matters: the development of student agency. Through rich, authentic assessments—not just pencil-and-paper tests—we can begin to understand the complexity of learning and human development: how students see themselves in the world (identity), their sense of connection to others (belonging), their ability to construct and defend original ideas (mastery), and their capacity to make a difference around what matters to them (efficacy). This more holistic approach is the compass that will guide us toward a next-generation model of data.

3. We need to start listening to stories at the margins of our schools and districts as the most valuable equity-centered data of all. This requires a mindset shift from data as something we extract from students and then organize into dashboards and reports toward a stance of deep listening and keen observation. We can collect valuable street data from students by conducting empathy interviews, shadowing students, and convening focus groups, student panels, and action research around problems of practice. Now more than ever, we need holistic, inclusive data that can only be gathered by going to the most marginalized families and students, listening deeply, and co-constructing recovery frameworks and priorities.

The solution to generational, systemic inequities will never emerge from a single source. When we continue to frame our analysis and approaches around standardized testing and other big data, we miss an opportunity to reimagine schooling for a post-pandemic world. Recovery isn’t just about academic test scores. It’s about building a next-generation approach to data and knowledge that tells the whole story—a story of loss, yes, but also of survival, resilience, new forms of learning, and yearning for a reimagined educational system. To dream this approach into existence, we need a data framework rooted in holistic assessment of student learning. In this brave new world, data will be humanizing, liberatory, and healing.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Professional Development Webinar
Strategies for Improving Student Outcomes with Teacher-Student Relationships
Explore strategies for strengthening teacher-student relationships and hear how districts are putting these methods into practice to support positive student outcomes.
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
Transform Teaching and Learning with AI
Increase productivity and support innovative teaching with AI in the classroom.
Content provided by Promethean
Curriculum Webinar Computer Science Education Movement Gathers Momentum. How Should Schools React?
Discover how schools can expand opportunities for students to study computer science education.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Data Opinion Why It’s So Tough to Get the Data Educators Want
Data dissatisfaction remains high. Harvard's Jon Fullerton tries to make sense of why that is.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Data Opinion How to Start Using Data to Achieve Equity for Students
Schools are awash with data but rarely do anything with it. One district started out by limiting the tools it uses.
Mackey Pendergrast & Erica Hartman
5 min read
Entry Point
Data Education Department to Expand Data Collection on COVID and Schools
The Education Department will expand the limited survey started last year to query about issues like learning loss and health precautions.
2 min read
Image of a data dashboard.
Suppachok Nuthep/iStock/Getty
Data 'Extreme' Chronic Absenteeism? Pandemic School Attendance Data Is Bleak, But Incomplete
Only a handful of states have reported information on chronic absenteeism, right as advocates warn that addressing it will be a top priority this fall.
4 min read
Image of an empty desk.
Laura Baker/Education Week and kazuma seki/iStock/Getty