Opinion
Data Opinion

Standardized Tests Aren’t the Only Meaningful Data on Student Achievement

The case for using “street data”
By Shane Safir — September 19, 2022 5 min read
conceptual image of a young person walking a trendline
  • Save to favorites
  • Print

Public education needs a new data paradigm. The past 20 years have trapped us in a narrow definition of data that peddles a deficit narrative about Black, Indigenous, Latinx students, students with learning needs, and other historically marginalized groups. This era has stripped educators of agency and made student voice peripheral to our accounting of “school progress,” which we largely measure through incremental gains in test scores and other quantitative metrics like attendance and graduation rates.

It’s time to reimagine “data” for the next generation of schooling, beginning with street-level qualitative data that is rooted in student experience and invites educators to become ethnographers rather than statisticians in their pursuit of educational equity.

After this month’s release of data from the National Assessment of Educational Progress showed significant dips in student math and reading scores, national coverage of the results has told a story of “learning loss” and the need to fill student “gaps.” This framing echoes nearly two decades of test-centered solutions that fail to account for the complexity of learning.

Complex times call for complex approaches, including an orientation toward a love of learning rather than the deficit-based loss: How can we help children—and teachers—cultivate joy in the wake of so much personal loss and tragedy? We must embrace an expansive concept of data, one that centers student and family voices as well as students’ cultural values, funds of knowledge, identity development, sense of belonging, and mastery of 21st century competencies that extend well beyond test-taking.

1. We need to adjust the role of big, quantitative data in school improvement efforts. The entire premise that we can define “learning loss” as a function of how students should have scored on standardized tests is problematic, dangerous, and dehumanizing. Achievement is about so much more than test scores, and authentic assessments of achievement would incorporate student voice, reflection, and performance on real-world learning tasks, not just tests.

When we only evaluate learning through such analysis as “the average high-poverty school that remained in remote instruction for a majority of 2020-21 lost roughly .44 standard deviations in achievement” (from the recent report out of Harvard University, “The Consequences of Remote and Hybrid Instruction During the Pandemic”), we pursue a path of uncreative solutions that will further deepen opportunity gaps and reproduce status quo approaches.

Author and leadership coach Jamila Dugan calls this overemphasis of large-scale, quantitative data the “boomerang equity trap": investing time and resources to understand your equity challenges but reverting back to previous mental models in ways that lead to unintentionally harmful solutions.

To suggest—as the Harvard University report does—that interventions like high-dosage tutoring or remediation will produce incremental gains (i.e., a .38 standard deviation gain in math) exhibits an incarceration of the imagination that simply doesn’t speak to parents and students. Ask any parent of school-age children what matters most to them in the wake of a traumatic global pandemic, and I guarantee they will not clamor for solutions measured by a .38 standard deviation gain in math.

This type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just "unplugged" for a while and lost valuable "processing" time.

This is not to say that increasing numeracy and literacy doesn’t matter; of course, it does. However, this type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just “unplugged” for a while and lost valuable “processing” time, the solution to which is to simply run them overtime with high-dosage tutoring to get them where they need to be.

2. We need to shift our focus from so-called “achievement gaps” to opportunity gaps by collecting qualitative information that is close to the learner. This qualitative information is what I call “street data.” Instead of fixating on filling purported “gaps” in learning, street data can allow education researchers to start addressing opportunity gaps, or what Gloria Ladson-Billings refers to as the lingering “education debt” owed to Black, Indigenous, and other historically marginalized groups of students.

This would mean building more robust and holistic data systems that highlight student and community voice—from school planning templates to report cards to district equity audits that don’t just include numbers but tell a story of change. Researchers should particularly focus on learning more from school districts that have succeeded in closing gaps and developing innovative approaches to supporting underresourced schools.

In contrast to quantitative data, street data offers a wider lens to start measuring what truly matters: the development of student agency. Through rich, authentic assessments—not just pencil-and-paper tests—we can begin to understand the complexity of learning and human development: how students see themselves in the world (identity), their sense of connection to others (belonging), their ability to construct and defend original ideas (mastery), and their capacity to make a difference around what matters to them (efficacy). This more holistic approach is the compass that will guide us toward a next-generation model of data.

3. We need to start listening to stories at the margins of our schools and districts as the most valuable equity-centered data of all. This requires a mindset shift from data as something we extract from students and then organize into dashboards and reports toward a stance of deep listening and keen observation. We can collect valuable street data from students by conducting empathy interviews, shadowing students, and convening focus groups, student panels, and action research around problems of practice. Now more than ever, we need holistic, inclusive data that can only be gathered by going to the most marginalized families and students, listening deeply, and co-constructing recovery frameworks and priorities.

The solution to generational, systemic inequities will never emerge from a single source. When we continue to frame our analysis and approaches around standardized testing and other big data, we miss an opportunity to reimagine schooling for a post-pandemic world. Recovery isn’t just about academic test scores. It’s about building a next-generation approach to data and knowledge that tells the whole story—a story of loss, yes, but also of survival, resilience, new forms of learning, and yearning for a reimagined educational system. To dream this approach into existence, we need a data framework rooted in holistic assessment of student learning. In this brave new world, data will be humanizing, liberatory, and healing.

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Science of Reading: Emphasis on Language Comprehension
Dive into language comprehension through a breakdown of the Science of Reading with an interactive demonstration.
Content provided by Be GLAD
English-Language Learners Webinar English Learners and the Science of Reading: What Works in the Classroom
ELs & emergent bilinguals deserve the best reading instruction! The Reading League & NCEL join forces on best practices. Learn more in our webinar with both organizations.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Data What the Research Says What Does 'Evidence-Based' Mean? A Study Finds Wide Variation.
Fewer than 1 in 3 education interventions get consistent judgments on their evidence base from reviewers.
5 min read
photograph of a magnifying glass on an open book
Valiantsin Suprunovich/iStock
Data 'Hidden Homeless': A Key Measure of Homelessness Excludes Most Students
Federal agencies differ in how they measure homelessness—and many vulnerable students are left out.
3 min read
Photograph of a low angle view of children with backpacks climbing the school staircase.
E+/Getty
Data Spotlight Spotlight on Leveraging Data for Student Success
This Spotlight will help you learn how data can help schools target resources, explore how to improve instruction with data, and more.
Data What Superintendents Say They Need More of to Help Them Manage Districts
98% of those surveyed said better data would make them more comfortable making decisions.
2 min read
Image of a data dashboard.
Suppachok Nuthep/iStock/Getty