Public education needs a new data paradigm. The past 20 years have trapped us in a narrow definition of data that peddles a deficit narrative about Black, Indigenous, Latinx students, students with learning needs, and other historically marginalized groups. This era has stripped educators of agency and made student voice peripheral to our accounting of “school progress,” which we largely measure through incremental gains in test scores and other quantitative metrics like attendance and graduation rates.
It’s time to reimagine “data” for the next generation of schooling, beginning with street-level qualitative data that is rooted in student experience and invites educators to become ethnographers rather than statisticians in their pursuit of educational equity.
After this month’s release of data from the National Assessment of Educational Progress showed significant dips in student math and reading scores, national coverage of the results has told a story of “learning loss” and the need to fill student “gaps.” This framing echoes nearly two decades of test-centered solutions that fail to account for the complexity of learning.
Complex times call for complex approaches, including an orientation toward a love of learning rather than the deficit-based loss: How can we help children—and teachers—cultivate joy in the wake of so much personal loss and tragedy? We must embrace an expansive concept of data, one that centers student and family voices as well as students’ cultural values, funds of knowledge, identity development, sense of belonging, and mastery of 21st century competencies that extend well beyond test-taking.
1. We need to adjust the role of big, quantitative data in school improvement efforts. The entire premise that we can define “learning loss” as a function of how students should have scored on standardized tests is problematic, dangerous, and dehumanizing. Achievement is about so much more than test scores, and authentic assessments of achievement would incorporate student voice, reflection, and performance on real-world learning tasks, not just tests.
When we only evaluate learning through such analysis as “the average high-poverty school that remained in remote instruction for a majority of 2020-21 lost roughly .44 standard deviations in achievement” (from the recent report out of Harvard University, “The Consequences of Remote and Hybrid Instruction During the Pandemic”), we pursue a path of uncreative solutions that will further deepen opportunity gaps and reproduce status quo approaches.
Author and leadership coach Jamila Dugan calls this overemphasis of large-scale, quantitative data the “boomerang equity trap": investing time and resources to understand your equity challenges but reverting back to previous mental models in ways that lead to unintentionally harmful solutions.
To suggest—as the Harvard University report does—that interventions like high-dosage tutoring or remediation will produce incremental gains (i.e., a .38 standard deviation gain in math) exhibits an incarceration of the imagination that simply doesn’t speak to parents and students. Ask any parent of school-age children what matters most to them in the wake of a traumatic global pandemic, and I guarantee they will not clamor for solutions measured by a .38 standard deviation gain in math.
This type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just "unplugged" for a while and lost valuable "processing" time.
This is not to say that increasing numeracy and literacy doesn’t matter; of course, it does. However, this type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just “unplugged” for a while and lost valuable “processing” time, the solution to which is to simply run them overtime with high-dosage tutoring to get them where they need to be.
2. We need to shift our focus from so-called “achievement gaps” to opportunity gaps by collecting qualitative information that is close to the learner. This qualitative information is what I call “street data.” Instead of fixating on filling purported “gaps” in learning, street data can allow education researchers to start addressing opportunity gaps, or what Gloria Ladson-Billings refers to as the lingering “education debt” owed to Black, Indigenous, and other historically marginalized groups of students.
This would mean building more robust and holistic data systems that highlight student and community voice—from school planning templates to report cards to district equity audits that don’t just include numbers but tell a story of change. Researchers should particularly focus on learning more from school districts that have succeeded in closing gaps and developing innovative approaches to supporting underresourced schools.
In contrast to quantitative data, street data offers a wider lens to start measuring what truly matters: the development of student agency. Through rich, authentic assessments—not just pencil-and-paper tests—we can begin to understand the complexity of learning and human development: how students see themselves in the world (identity), their sense of connection to others (belonging), their ability to construct and defend original ideas (mastery), and their capacity to make a difference around what matters to them (efficacy). This more holistic approach is the compass that will guide us toward a next-generation model of data.
3. We need to start listening to stories at the margins of our schools and districts as the most valuable equity-centered data of all. This requires a mindset shift from data as something we extract from students and then organize into dashboards and reports toward a stance of deep listening and keen observation. We can collect valuable street data from students by conducting empathy interviews, shadowing students, and convening focus groups, student panels, and action research around problems of practice. Now more than ever, we need holistic, inclusive data that can only be gathered by going to the most marginalized families and students, listening deeply, and co-constructing recovery frameworks and priorities.
The solution to generational, systemic inequities will never emerge from a single source. When we continue to frame our analysis and approaches around standardized testing and other big data, we miss an opportunity to reimagine schooling for a post-pandemic world. Recovery isn’t just about academic test scores. It’s about building a next-generation approach to data and knowledge that tells the whole story—a story of loss, yes, but also of survival, resilience, new forms of learning, and yearning for a reimagined educational system. To dream this approach into existence, we need a data framework rooted in holistic assessment of student learning. In this brave new world, data will be humanizing, liberatory, and healing.
