School & District Management

Schools Struggle to Use Data to Spark Improvement

By Benjamin Herold — March 20, 2018 | Corrected: March 21, 2018 7 min read
  • Save to favorites
  • Print

Corrected: A previous version of this article misspelled the name of Menomonee Falls School District. The name of the Carnegie Foundation for the Advancement of Teaching has also been corrected.

Can educational data be used to actually help schools get better?

That’s the hope behind a push to bring to K-12 schools a management philosophy known as “continuous improvement” that has flourished in fields such as health care, manufacturing, and social services.

But experts contend that the K-12 education system’s current data infrastructure, built in response to the federal No Child Left Behind Law and focused primarily on holding schools accountable, looms as a significant barrier.

“New data systems really have to allow [teachers and principals] to say what they want to accomplish, and determine whether they’ve accomplished it,” said Louis Gomez, a professor of education and information studies at the University of California, Los Angeles, and a co-author of the 2015 book, Learning to Improve, How America’s Schools Can Get Better at Getting Better. “Just monitoring how we’re doing is not going to help us get better.”

Examples of what an improvement-based data infrastructure actually looks like in practice are few and far between.

Louis GomezUCLA Professor

Wisconsin’s 4,000-student Menomonee school district is lauded for using a comprehensive approach to steadily improving everything from recess safety to the cleanliness of its classrooms, but educators and administrators there have relied on mostly informal systems for collecting and analyzing information.

The 400,000-student Chicago school district has been at the forefront of efforts to use ‘early-warning’ data to keep high school students on track to graduation and college, but such work typically relies on existing data systems to support a narrowly targeted improvement project. And the New York-based nonprofit New Visions for Public Schools has taken the lead on developing brand new data systems designed to help schools collect and track the kinds of “process data” that continuous-improvement proponents describe as critical, but such tools remain on the fringes of the K-12 sector.

Far more typical are districts like Dallas, where budget woes and leadership churn have hampered efforts to build and use data systems in new ways.

See Also: A Primer on Continuous School Improvement

As a field, education has “placed tremendous emphasis on data for accountability purposes,” said Mark Dunetz, the president of New Visions for Public Schools. “We’ve done a much poorer job at providing practitioners with the tools they need to manage the types of tasks that are the day-to-day bread and butter of schools, and that enable everything else to happen.”

Little Impact on Teaching

The idea of schools making better use of educational data is nothing new.

Back in 2010, the U.S. Department of Education described data-driven decision making as a “national priority,” saying that states and school districts had made significant progress in building a new data infrastructure.

But in a report that year titled “Use of Education Data at the Local Level: From Accountability to Instructional Improvement,” the department also concluded that such work was having little impact on actual classroom teaching.

The reasons?

“School staffs’ perception of barriers to greater use of data include a sense of lack of time, system-usability issues, the perception that the data in the system are not useful, and district policies around curriculum coverage or pacing that prohibit modifying learning time to match student needs,” the report concludes.

Fast forward eight years, and the lingo has changed, but such challenges are still endemic, said Andrew Krumm, the director of learning analytics at Digital Promise, a nonprofit that promotes the use of technology to improve schools.

Andrew KrummData Researcher, Digital Promise

Too often, Krumm and other proponents of continuous improvement say, schools get caught in a cycle in which administrators and policymakers identify a problem; develop a program or intervention to address that problem; roll it out; and try to determine whether or not it worked. The data used to measure success is typically a long-term outcome measure, such as standardized test results. If scores go up, administrators try to scale the program across other schools and districts—often with little consideration for why the program worked, or how conditions on the ground in other schools and districts might vary.

That approach amounts to “implementing fast and learning slow,” Gomez and his co-authors from the Carnegie Foundation for the Advancement of Teaching argue in their 2015 book.

Instead, they argue, schools should be focused on the reverse: implementing improvement strategies more methodically, but learning much more about them as they happen.

Ideally, they say, such a process would entail identifying the problem schools want to fix; developing a theory about how to improve it; and then helping the people closest to the problem—usually teachers, principals, and other school staff—to develop measures of day-to-day progress that are aligned to that theory. Technology tools should help schools monitor three things: whether they’re actually doing what they set out to do, whether it’s making a difference on the measures that educators developed locally, and how such efforts impact the kinds of long-term outcomes measures that are typically used now.

“The most interesting work happens when schools supplement system-wide data with finer-grained, day-in-and-day-out data,” Krumm said in an interview. “We need to be collecting data that is much closer to what we’re actually trying to change.”

Tracking Daily Decisions

That’s generally the approach that New Visions for Public Schools is trying to take with the custom educational software it’s building.

The nonprofit currently provides its data tools to 342 schools in New York City, including 10 charter schools that it directly operates and several dozen district schools for which the group serves as a “comprehensive lead partner.”

The focus is on closely tracking the seemingly mundane decisions that principals, assistant principals, teachers, and guidance counselors have to make each day, then getting them the information they need to make those decisions.

Graduation planning is one big example. Additional functions aim to help counselors match students to internships and other opportunities in the most efficient way possible, or to help administrators track student attendance and participation in after-school activities.

“I think we’ve historically underestimated the complexity of work in schools, where there is a tremendous volume of decisions that need to be made, and the outcomes are often determined by our ability to be consistent,” Dunetz said. “If you don’t know whether a classroom had the right textbooks, or whether the computers worked, or if kids showed up, it’s very difficult to draw any conclusions about whether what you’re doing is working, and why.”

Notably, though, Dunetz said New Visions determined it had to build that kind of software from scratch. Very few software applications allow schools real flexibility to decide what data are most important to them, then collect and analyze that information customized ways.

In theory, at least, the new technologies that have flooded into schools over the past decade could help. Countless digital tools are now part of students’ everyday learning, and many are capable of generating reams of data on everything that a student (or teacher) does.

But much of that software still serves accountability purposes, Krumm said. The experience of the 157,000-student Dallas school system highlights the challenges on the ground to building new data systems to support continuous improvement work.

Back in 2009, the Gates Foundation gave the Dallas district a three-year, $3.8 million grant to strengthen its “college readiness warning system,” modeled after Chicago’s efforts. The aim was to help teachers use data, pulled primarily from existing student-information systems, to “identify student needs, provide appropriate interventions, and ultimately increase college readiness.”

The tool the district developed was “a dashboard pulling up live data” on such indicators as student attendance and access to financial-aid counseling, said Cecilia Oakeley, who started with the district in 2005 and is now an assistant superintendent for evaluation and assessment.

That system took a while to build, Oakeley said. Some schools used it more than others.

And then the grant ran out, the superintendent left, and budget cuts hit.

“The platform went by the wayside,” she said. “When there are switches in administration, some work gets lost in the shuffle.”

Fortunately, Oakeley said, all was not lost. The district had also built a supplemental data system to allow school staff to also examine other relevant information, such as students’ grades in core subject areas. District-level staff still use that system to monitor how many students are on track for graduation at the end of 9th grade. And former superintendent Michael Hinojosa, who led the original push for an early-warning system in Dallas before moving on, is now back leading the district, bringing renewed attention to efforts to get school-level staff to use the information, too.

But for now, at least, that’s still a far cry from the type of continuous-improvement data systems that proponents envision.

“This is a process,” said Gomez of UCLA. “People are coming to understand that all these [data systems we have now] were not created with improvement in mind. This is a first and important step.”

Coverage of continuous-improvement strategies in education is supported in part by a grant from the Bill & Melinda Gates Foundation, at Education Week retains sole editorial control over the content of this coverage.
A version of this article appeared in the March 21, 2018 edition of Education Week as Schools Struggle to Use Data to Get Better


Student Well-Being K-12 Essentials Forum Boosting Student and Staff Mental Health: What Schools Can Do
Join this free virtual event based on recent reporting on student and staff mental health challenges and how schools have responded.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
Practical Methods for Integrating Computer Science into Core Curriculum
Dive into insights on integrating computer science into core curricula with expert tips and practical strategies to empower students at every grade level.
Content provided by

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Late Arrivals, Steep Costs: Why Some Districts Ditch Third-Party Bus Companies
Districts are facing a host of transportation challenges. Some have addressed them by deciding to bring buses back in house.
6 min read
School buses parked in Helena, Mont., ahead of the beginning of the school year on Aug. 20, 2021.
Some districts are pulling back on decisions to outsource bus services in an effort to save money and improve service.
Iris Samuels/AP
School & District Management Rising Tensions From Israel-Hamas War Are Seeping Into Schools
As effects of the war reverberate in school communities, schools have federal responsibilities to create discrimination-free environments.
5 min read
People gather in Pliny Park in Brattleboro, Vt., for a vigil, Monday, Nov. 27, 2023, for the three Palestinian-American students who were shot while walking near the University of Vermont campus in Burlington, Vt., Saturday, Nov. 25. The three students were being treated at the University of Vermont Medical Center, and one faces a long recovery because of a spinal injury, a family member said.
People gather in Pliny Park in Brattleboro, Vt., for a vigil, Monday, Nov. 27, 2023, for the three Palestinian-American students who were shot while walking near the University of Vermont campus in Burlington, Vt., Saturday, Nov. 25. Tensions over the Israel-Hamas war are playing out in schools and colleges across the country, including some K-12 schools.
Kristopher Radder/The Brattleboro Reformer via AP
School & District Management The Missed Opportunity for Public Schools and Climate Change
More cities are creating climate action plans, but schools are often left out of the equation.
4 min read
Global warming illustration, environment pollution, global warming heating impact concept. Change climate concept.
Collage by Gina Tomko/Education Week and iStock/Getty Images Plus
School & District Management 13 States Bar School Board Members From Getting Paid. Here's Where It's Allowed (Map)
There are more calls to increase school board members' pay, or to allow them to be paid at all.
Two professional adults, with a money symbol.