Opinion
Student Achievement Opinion

Standardized Tests Aren’t the Only Meaningful Data on Student Achievement

The case for using “street data”
By Shane Safir — September 19, 2022 5 min read
conceptual image of a young person walking a trendline
  • Save to favorites
  • Print

Public education needs a new data paradigm. The past 20 years have trapped us in a narrow definition of data that peddles a deficit narrative about Black, Indigenous, Latinx students, students with learning needs, and other historically marginalized groups. This era has stripped educators of agency and made student voice peripheral to our accounting of “school progress,” which we largely measure through incremental gains in test scores and other quantitative metrics like attendance and graduation rates.

It’s time to reimagine “data” for the next generation of schooling, beginning with street-level qualitative data that is rooted in student experience and invites educators to become ethnographers rather than statisticians in their pursuit of educational equity.

After this month’s release of data from the National Assessment of Educational Progress showed significant dips in student math and reading scores, national coverage of the results has told a story of “learning loss” and the need to fill student “gaps.” This framing echoes nearly two decades of test-centered solutions that fail to account for the complexity of learning.

Complex times call for complex approaches, including an orientation toward a love of learning rather than the deficit-based loss: How can we help children—and teachers—cultivate joy in the wake of so much personal loss and tragedy? We must embrace an expansive concept of data, one that centers student and family voices as well as students’ cultural values, funds of knowledge, identity development, sense of belonging, and mastery of 21st century competencies that extend well beyond test-taking.

1. We need to adjust the role of big, quantitative data in school improvement efforts. The entire premise that we can define “learning loss” as a function of how students should have scored on standardized tests is problematic, dangerous, and dehumanizing. Achievement is about so much more than test scores, and authentic assessments of achievement would incorporate student voice, reflection, and performance on real-world learning tasks, not just tests.

When we only evaluate learning through such analysis as “the average high-poverty school that remained in remote instruction for a majority of 2020-21 lost roughly .44 standard deviations in achievement” (from the recent report out of Harvard University, “The Consequences of Remote and Hybrid Instruction During the Pandemic”), we pursue a path of uncreative solutions that will further deepen opportunity gaps and reproduce status quo approaches.

Author and leadership coach Jamila Dugan calls this overemphasis of large-scale, quantitative data the “boomerang equity trap": investing time and resources to understand your equity challenges but reverting back to previous mental models in ways that lead to unintentionally harmful solutions.

To suggest—as the Harvard University report does—that interventions like high-dosage tutoring or remediation will produce incremental gains (i.e., a .38 standard deviation gain in math) exhibits an incarceration of the imagination that simply doesn’t speak to parents and students. Ask any parent of school-age children what matters most to them in the wake of a traumatic global pandemic, and I guarantee they will not clamor for solutions measured by a .38 standard deviation gain in math.

This type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just "unplugged" for a while and lost valuable "processing" time.

This is not to say that increasing numeracy and literacy doesn’t matter; of course, it does. However, this type of myopic analysis of standardized test scores at the expense of other meaningful sources of data treats students like computers that were just “unplugged” for a while and lost valuable “processing” time, the solution to which is to simply run them overtime with high-dosage tutoring to get them where they need to be.

2. We need to shift our focus from so-called “achievement gaps” to opportunity gaps by collecting qualitative information that is close to the learner. This qualitative information is what I call “street data.” Instead of fixating on filling purported “gaps” in learning, street data can allow education researchers to start addressing opportunity gaps, or what Gloria Ladson-Billings refers to as the lingering “education debt” owed to Black, Indigenous, and other historically marginalized groups of students.

This would mean building more robust and holistic data systems that highlight student and community voice—from school planning templates to report cards to district equity audits that don’t just include numbers but tell a story of change. Researchers should particularly focus on learning more from school districts that have succeeded in closing gaps and developing innovative approaches to supporting underresourced schools.

In contrast to quantitative data, street data offers a wider lens to start measuring what truly matters: the development of student agency. Through rich, authentic assessments—not just pencil-and-paper tests—we can begin to understand the complexity of learning and human development: how students see themselves in the world (identity), their sense of connection to others (belonging), their ability to construct and defend original ideas (mastery), and their capacity to make a difference around what matters to them (efficacy). This more holistic approach is the compass that will guide us toward a next-generation model of data.

3. We need to start listening to stories at the margins of our schools and districts as the most valuable equity-centered data of all. This requires a mindset shift from data as something we extract from students and then organize into dashboards and reports toward a stance of deep listening and keen observation. We can collect valuable street data from students by conducting empathy interviews, shadowing students, and convening focus groups, student panels, and action research around problems of practice. Now more than ever, we need holistic, inclusive data that can only be gathered by going to the most marginalized families and students, listening deeply, and co-constructing recovery frameworks and priorities.

The solution to generational, systemic inequities will never emerge from a single source. When we continue to frame our analysis and approaches around standardized testing and other big data, we miss an opportunity to reimagine schooling for a post-pandemic world. Recovery isn’t just about academic test scores. It’s about building a next-generation approach to data and knowledge that tells the whole story—a story of loss, yes, but also of survival, resilience, new forms of learning, and yearning for a reimagined educational system. To dream this approach into existence, we need a data framework rooted in holistic assessment of student learning. In this brave new world, data will be humanizing, liberatory, and healing.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Harnessing AI to Address Chronic Absenteeism in Schools
Learn how AI can help your district improve student attendance and boost academic outcomes.
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Spark Minds, Reignite Students & Teachers: STEM’s Role in Supporting Presence and Engagement
Is your district struggling with chronic absenteeism? Discover how STEM can reignite students' and teachers' passion for learning.
Content provided by Project Lead The Way
Recruitment & Retention Webinar EdRecruiter 2025 Survey Results: The Outlook for Recruitment and Retention
See exclusive findings from EdWeek’s nationwide survey of K-12 job seekers and district HR professionals on recruitment, retention, and job satisfaction. 

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Achievement Tutors Don't Get Much Training. A New Effort Could Help
For intensive tutoring to reach its potential, tutors may need more training to work with students and teachers.
3 min read
High school tutor giving male student one to one tutoring at a desk
iStock/Getty Images Plus
Student Achievement How Motivated Are Students to Drive Their Own Learning?
An international test of students in more than 80 countries and economies finds that many struggle with motivation.
4 min read
Unhappy young African American hipster guy in eyeglasses looking in distance, feeling bored preparing for examination or doing high school research project on computer, sitting at table in library.
iStock/Getty Images
Student Achievement Spotlight Spotlight on MTSS
This Spotlight explores key aspects of MTSS implementation, including its relationship to special education and effectiveness in improving student outcomes.
Student Achievement This District Provided Tutoring to Thousands of Students. The Results Were Mixed
A new study suggests that tutoring at scale could have a smaller impact than advocates had hoped.
6 min read
Waist-up view of early 30s teacher sitting with 11 year old Hispanic student at library round table and holding book as she pronounces the words.
E+