By Elana Karopkin-Gold, the senior strategist at Springpoint, and Michele Phillips, a consultant for Springpoint
Stronger data practices can help leaders better utilize data as a way to deeply understand the students they serve.
“Data-based decisionmaking.” “Data-driven instruction.” These are now-familiar terms in the world of education. An emerging understanding of the power of data across various sectors continues to excite savvy and creative educators who see value in leveraging data-driven design. In the past decade or so, many schools have invested in practices such as interim assessments to help teachers and leaders monitor and respond to trends in data. Some—mostly larger—districts have even invested in processes such as school report cards and ways of measuring “value-added.” While there is not yet full-sector clarity and alignment regarding where an investment in data-based practices yields the strongest results, there is no question that schools better serve students when they understand the students they serve.
We take this maxim to heart at Springpoint, a national nonprofit that supports districts, networks, and charters to design innovative new high school models. We encourage our partners to adopt a curiosity mindset; to constantly ask “why” and seek deep insights about their students as a first step in the school design journey.
Using a wide variety of quantitative and qualitative data helps leaders gain a true understanding of the young people they serve, which in turn allows them to articulate students’ most critical needs and assets, and devise important programmatic responses in the design of school—especially for the most marginalized students. Often, this requires leaders to both define data broadly and set up strategic systems and inflection points to collect and apply data in their iterative school design work. Strong school models are developed when leaders use both quantitative data—such as assessments, test scores, and demographic information—and qualitative data, such as community and student surveys, focus groups, in-depth interviews.
There is a perception that qualitative data is useful, but extremely hard to gather and analyze; in our work with schools and districts, we have found that gathering and manipulating quantitative data can be just as challenging. This article focuses on the barriers to collecting, analyzing, and using quantitative data in service of school design. We also discuss how districts and schools might start to address some of the technical challenges inherent in doing so.
Understanding Root Causes of Student Performance
Using data strategically and effectively allows leaders to leverage authentic opportunities that address students’ interests, strengths, and needs—and lets them continuously ensure that resource allocation and learning experiences are responsive to those specific needs and interests, as opposed to an assumed need or a fleeting education fad. For example, in our work supporting school designers to develop school models for students who are off-track to graduation as part of the Barr Foundation’s Engage New England initiative, we have seen that when data are not readily available and easy to manipulate and digest, it can be hard to assess whether students are off-track because of isolated skill gaps, challenges with English-language acquisition, chronic absenteeism, or something else. Without access to raw data and the ability to manipulate the data in service of testing hypotheses, there is little to do other than to make an assumption that all of these challenges are present in equal parts—or that they are so highly interrelated that they cannot be teased apart. This level of analysis is crude and does not allow school designers to understand key root causes, define the most critical solutions, and create learning environments that set students up for success.
Often a deeper understanding of students comes from cycles of inquiry in which leaders cut raw data in different ways, which allows educators to reject erroneous hypotheses and come to increasingly more nuanced and accurate conclusions. Without access to or aptitude with raw data, leaders are left relying on anecdotal or surface-level trends that can often misdiagnose the root causes of poor student performance. Such a misdiagnosis can point leaders toward solutions that demand time and energy to devise and operationalize without actually addressing the most critical needs.
There also exist limitations on how districts represent data when they focus narrowly on state reporting functions, rendering data cuts tailored to this purpose far less useful than they have the potential to be. Finally, we have seen leaders struggle with unruly data systems for reasons that run the gamut but often stem from the sheer number of platforms and tools, compounded by a lack of compatibility between them.
The following are some of the specific areas we have seen as obstacles in our work with practitioners at the school and district level:
Challenge #1: Access to and Aptitude With Raw Data
Quite often, student-level data files are only fully accessible at the district level and not by individual school leaders and staff. Even when accessible, data files may be unwieldy and available only through elaborate data portals. District data teams often understand their role as primarily providing information necessary for local, state, and federal accountability reporting. District data personnel are therefore more frequently focused on ensuring data are clean and ready for state reporting purposes than analyzing and packaging data for school-based personnel. Many school leaders would like to use data as a way to make decisions, but they do not always have the skills to use specific data tools or know-how to draw conclusions, refine a hypothesis, and articulate meaningful research questions. District data teams’ roles rarely allow them to provide sufficient school-based training or prepare school-facing data reports. Therefore, even when school-based personnel have access to raw files, few have enough training in data practices to extract what they need.
Data reporting and collection systems often evolve and change, and states or districts sometimes decide to switch vendors. Modifications such as these mean that data specialists at the district-level must spend a bulk of their time getting retrained in new systems and reporting functions. Another demand on their time may include needing to turnkey new information to school-based staff, but only for reporting purposes. There then remains little time for inquiry cycles that extract meaningful, informative data on student performance. As a result, school-based staff often become frustrated and left without usable school-level data to inform their practices or shape instructional and programmatic shifts.
Challenge #2: State Reporting Limitations, Especially for Vulnerable Populations
When data are collected exclusively for use by the state, the information is cut and analyzed according to state requirements, which may leave important gaps especially pertaining to the most vulnerable subpopulations of students. For example, state-reported data might not include subgroups of demographic data when the number of students falls below a required sample size. Leaders, therefore, cannot identify or address significant trends. Say a school experiences an uptick in enrollment of English-language learners but does not exceed the minimum threshold in a given year (e.g., 10 or 20 students per cohort), then year-to-year data will not be available regarding this group’s attendance, graduation, and state assessment outcomes. It may also be that important subgroups at a school are not defined in a traditional demographic sense. Natural groupings of students with similar characteristics and similar support needs may not be immediately apparent, such as previously incarcerated students, recent immigrants, students who are the breadwinners for their family, etc. Leaders need to ask the right questions and go beyond quantitative data to better understand who they are designing for. And finally, new, merged, or renamed schools may not have consistent reporting year-to-year because of changing school codes. Challenges such as these can render state-reported data inaccessible or inaccurate, making it harder to access data trends for students who most need visibility and support.
Challenge #3: Systems Overload
An increasingly pertinent issue in the education world is the lack of data interoperability—wherein systems are not set up to speak to one another. Even when systems are designed to make data gathering and analysis easier, inconsistent use can compound challenges. For example, schools often use multiple systems—perhaps there is one system for overall student-enrollment information (SIS); one for benchmark or interim assessments; one for participation in state assessments; possibly a separate system for grading, attendance, or behavioral-data tracking; and, increasingly, learning management systems that may or may not connect directly to the school’s SIS. Lack of data interoperability renders data-based decisionmaking arduous and unrealistic. While there are both local bright spots and nationally focused companies and initiatives—like Project Unicorn, Ed-fi Alliance, and Illuminate Education—that continue to work toward solutions, data-interoperability challenges still cause confusion and frustration, all while taking up leaders’ precious time.
These are not insignificant challenges, but solutions exist that could provide leaders with a baseline set of conditions for a student-centered, data-driven process.
Solution #1: Redefining the Work of District Data Teams
Districts must expand the vision of what their data teams can and should do beyond state reporting requirements. When districts cultivate a shared belief in the importance of getting data into the hands of school leaders, these data can be used as the basis of making and evaluating important decisions. In order to do this, districts need to build capacity for this work. That can include training programs as well as the direct development of key data cuts and analysis for leaders to leverage and employ. Through trainings and other engagements, district data experts can help schools understand that data-based practices are a not once-a-year occurrence. They can demonstrate why data-based cycles must be embedded in school practices throughout the year with a comprehensive understanding of what data are, how to collect and analyze data, and when to use the information to inform decisions and assess progress toward goals.
Solution #2: Systems Fixes
Because the very systems that are designed to support data use often make it challenging, leaders can develop standard operating requirements for data systems and school-based data entry. District data experts can play an important liaison role between systems providers and schools, ensuring that key systems speak to one another and school leaders can extract the data they most need. Before implementing or purchasing a new system, districts must consider how the new product will interconnect with other systems. When possible, new products should reduce the overall number of unique systems to avoid overload, minimize training needs, and increase ease of access and usability of data.
Solution #3: Finding the Right Data
Along with access to and aptitude with data, leaders need to be able to ask the right questions that can surface useful answers, think creatively about how to source information, approach analyses from different angles to yield actionable insights, and develop the muscle that will allow them to apply the valuable information surfaced during inquiry cycles. At Springpoint, we conduct research visits alongside leaders, demonstrating how to execute robust inquiry cycles as a way to gather qualitative and quantitative data, which can inform nuanced conclusions that lead to community-specific solutions. We encourage and support leaders to be strategic about the grain size of their research questions within each inquiry cycle since that will help them find patterns, variation, and bright spots. We help build leaders’ capacity to iterate on school designs. Districts interested in supporting data-based practices need to ensure they cultivate the mindsets of curiosity and the skills of data analysis, and they need to allocate sufficient time for this in trainings and other professional-development experiences. School leaders and teams can then track student experiences and school structures that they believe will lead to strong student outcomes, see the results, refine, and repeat. These practices allow leaders to understand the real impact of their choices and give them the necessary information to course correct when needed.
* * *
Effective data practices have the power to help districts and schools be more aware of trends in student performance. Once school leaders can see important patterns, they can be more strategic, intentional, and responsive. The existence of foundational data systems and practices represents a crucial first step, but schools also need training and support from the district level, easy access to the most important data sets and data cuts, and effective and efficient systems that speak to each other. Critical data practices, when done well, can elucidate a true understanding of students and their needs, which in turn fuels effective school design and continuous model improvement.
Photos, courtesy of Springpoint, from the top:
- A high school student from Brooklyn, New York
- High school students engaged in a writing class
- High school staff in Brooklyn, NY during an afternoon meeting. Creating space and time for adults to collaborate is a key way to allow them to review and use student data to design around students.