In the Fulton County, Ga., public school system in Atlanta, it’s not enough to think a strategy is working—you have to have the data to back it up.
“The data speaks in Fulton,” says the 88,000-student district’s executive director of continuous improvement, Dana McGraw. “That’s our foundation.”
In fact, the district is so well-known for its data-management system and data-driven decisionmaking techniques that it has been named a “best practice” district by both the Washington-based Consortium for School Networking’s Data-Driven Decision Making Initiative and the education division of the Houston-based American Productivity and Quality Center, a nonprofit organization that helps schools improve performance through better use of data and benchmarking.
But the evolution into a data-oriented culture in Fulton County did not happen overnight, says Martha Greenway, the deputy superintendent of organizational advancement for the district. Over the past 10 years, a combination of focused leadership, an eye toward continuous improvement, and an emphasis on open communication between departments in the district has led to a culture of data that focuses on putting usable information into the hands of those closest to instruction—the teachers.
Previously, “the only way [teachers] could get to the data was for the central staff to run reports for them,” says Greenway. But by integrating student achievement data into the district’s student information system, teachers are now able to access data on all past students as well as the students currently assigned to them.
“That [step] leapfrogged us over into getting people to actually use the data,” says Greenway.
Teachers can now put their own assessments into the data system and run their own daily reports to inform their instruction.
Part of what helps keep each school, as well as teachers, on track is the Balanced Scorecard, a system implemented in the district in 2001 that defines specific district-wide targets and benchmarks, which are then used to create individualized objectives for each school.
Each of the measures defined in the Balanced Scorecard is based on the level of student achievement as defined by assessments as well as demographic data to target specific populations for improvement, says Greenway.
“The Balanced Scorecard we just embraced as a system,” says Katie Reeves, a member of the Fulton County Board of Education. “I’m from a business background, and it certainly appealed to my business nature.”
It has “allowed our teachers to use the information, and not just as an end-of-the-year, target instruction [tool],” says Reeves. “It’s a management tool that survives different board members, different superintendents.”
‘Fear of Data’
Implementing the Balanced Scorecard system was a marked change from the way that performance had previously been measured in the Fulton County district, and it took some time for teachers to adjust, says Greenway.
Previously, it was difficult to define targets because of the lack of consistency in the data available, says Greenway. Achievement was measured by standardized tests, but not every grade level or subject area was tested, so it was difficult to analyze performance, she says.
“People are very anxious if they feel like they’re being held accountable for something when they don’t have the resources and skills,” she says. But once teachers became comfortable with the system and learned more about how it could be used, they embraced it, says Greenway.
“For the first time, [teachers] could see data in an integrated way, which gave them the ability to easily judge whether their students were on track,” she says. “The focus on [providing data] at the teacher’s level put the emphasis on improving instruction and making the teacher’s work more efficient and effective.”
Elizabeth M. Laird, the program manager for the Austin, Texas-based Data Quality Campaign, a nationwide nonprofit organization designed to encourage the use of data in education, agrees that an initial shift to a more data-centric approach can be intimidating for teachers at first.
“Having a fear of data is completely rational, but that cultural change is underway in education,” she says. “[Data] is not just something that’s used to rank schools or teachers, but something that really can help them do their job.”
In order to help teachers see that connection, district leaders have to be “clear about how the data will be used in a way that supports improving student achievement,” says Laird.
The passage of the federal No Child Left Behind Act in 2001 sparked a renewed focus on collecting and reporting data, and advancements in technology have dovetailed with that emphasis, says Jane A. Lockett, a retired IBM consultant specializing in the education industry and business intelligence.
“Now we’re starting to implement ... intelligence systems that are able to harvest data, combine it, or integrate it, and that really gives a very personal and customized view of how a teacher’s students look,” she says. “We’re finally getting to the point where systems are being used that help customize and personalize teaching and learning.”
As a result, school officials should now be asking themselves, “What is the true data we need to collect? What are the questions we’re trying to ask and the answers we hope to find?” says Lockett.
Better use of data can also reduce the number of interpersonal issues within a district, says Diane Kline, the senior project manager for APQC Education.
“When you have the data in front of you, the discussion becomes about the data versus about the people, and that does a number of things,” she says. “One is it kind of cuts down on any sort of defensive or interpersonal issues that you might run into, and it makes the discussions so much more valid because it’s not based on opinions or hunches.”
School Board Leadership
In addition to using the data-management system to help improve student achievement, the system has also been tapped to ensure that the district’s operations are running as efficiently and effectively as possible, says Greenway.
The amount of data collected about the demographics and characteristics of Fulton County public schools allows school officials to compare the district with other, similar systems. For example, the data-management system allows school officials to compare Fulton’s transportation system with that of a similarly sized district to see if there might be a better, more efficient way of transporting students.
By benchmarking Fulton against other districts, changes can be made quickly to improve and streamline processes, says McGraw, the executive director of continuous improvement. “We don’t have time, in education, to recreate the wheel,” she says. “You’ve got to be willing to say, ‘We’re not perfect. Maybe there’s a better way to do it.’ ”
Despite having a high turnover of superintendents in recent years—the district went through five superintendents between 2000 and 2005—data has remained a high priority for the board of education, says Superintendent Cindy Loe, who has been in that position about a year. “What really sets the district apart from others that I’ve seen that have had a lot of changes in superintendent leadership is that the board is very stable,” she says.
And when choosing new superintendents, the board purposely looks for someone who has a commitment to data and continuous improvement, says Loe.
“What’s interesting is the board is constantly looking to improve and build on the data-management system, and as they hire new superintendents ... they are constantly looking for a superintendent that is not only going to accept the philosophy that the school system is going to be data-driven, [but] they’re also looking for one who will take that data-driven thinking and bring it to the next, higher level by adding or enhancing it with additional data analysis measures,” she says.
‘Deliberate and Logical’
That emphasis has led to “a culture [of data in Fulton],” says Katie Lovett, the project director of the Data-Driven Decision Making Initiative at the Consortium for School Networking and a former chief information officer of the Fulton County public schools. “The culture does not happen overnight, and it does not happen without strong leadership.”
“If the school district doesn’t make the commitment, not only to the technology, but for using the data” in a day-to-day way, the data system won’t be effective, says Lovett.
Superintendent Loe has brought an emphasis on “value-added analysis” to the district.
“I would like to enhance the data-management tools by including a focus on value-added analysis where we are really trying to determine what value each school or department is adding to the students,” she says. Determining how much value has been added to a certain group of students or a department requires a more longitudinal look at the data, rather than a snapshot, says Loe.
Loe has also pushed for cross-divisional action teams, which gather members from different departments to utilize data to tackle an issue—such as figuring out what textbooks to use, maximizing the gifted and talented programs, or examining the district’s health insurance.
The cross-divisional action teams “break down the bureaucracy that can happen in a large organization,” says Loe. “Every voice is heard.”
In addition, the teams are asked to make recommendations on their respective topics within 90 days—a requirement designed to help move those suggestions forward.
“Sometimes we have a tendency to kind of study things to death and nothing really gets done,” says Loe. But a set deadline that requires a quick turnaround of information can help spur action on the issues, she says.
Continuing to improve the data-management system to boost student achievement and streamline operations is the backbone of Fulton’s data-driven decisionmaking, says Greenway, the deputy superintendent of organizational advancement.
“There has been this common vision that we are building a data-driven organization that’s focused on continuous process improvement, and these are all the building blocks we have to put in place” to achieve that goal, she says. “It’s just been very deliberate and logical. One step after the other.”
