In the Fulton County, Ga., public school system, it’s not enough to think a strategy is working—you have to have the data to back it up.
“The data speaks in Fulton,” said Dana McGraw, the executive director of continuous improvement for the 88,000-student district, which straddles the city of Atlanta. “That’s our foundation.”
In fact, the district is so well known for its data-management system and data-driven-decisionmaking techniques that it has been named a “best practice” district by both the Washington-based Consortium for School Networking’s Data-Driven Decision Making Initiative and the education division of the Houston-based American Productivity and Quality Center, a nonprofit group that helps schools improve performance through better use of data and benchmarking.
Fulton County is one of various school districts across the country that have made data-driven decisionmaking a top priority.
Others include the 140,000-student Montgomery County schools in Maryland, the 13,000-student Community Consolidated School District #15 in Palatine, Ill., and the 3,400-student Western Heights district in Oklahoma City. Western Heights’ superintendent, Joe Kitchens, was recently named the 2008 Data Leader of the Year by the Austin, Texas-based Data Quality Campaign, a nationwide nonprofit organization designed to encourage the use of data in education. (“Leading the Charge for Real-Time Data,” June 10, 2009.)
Help for Teachers
The development of a data-oriented outlook in Fulton County did not happen overnight, said Martha Greenway, the district’s deputy superintendent for organizational advancement. Over the past 10 years, a combination of focused leadership, an eye toward continuous improvement, and an emphasis on open communication between departments in the district has led to a culture that focuses on putting usable information into the hands of those closest to instruction—the teachers.
Previously, “the only way [teachers] could get to the data was for the central staff to run reports for them,” Ms. Greenway said. But with the integration of achievement data into the district’s student-information system, teachers are now able to access data on all past students as well as those currently assigned to them.
“That [step] leapfrogged us over into getting people to actually use the data,” Ms. Greenway said.
Teachers can now put their own assessments into the data system and run their own daily reports to inform their instruction.
Part of what helps keep each school, as well as teachers, on track is the Balanced Scorecard, a system implemented in the district in 2001 that defines specific districtwide targets and benchmarks, which are then used to create individualized objectives for each school.
Each of the measures defined in the Balanced Scorecard is based on the level of student achievement as identified by assessments as well as demographic data to target specific populations for improvement, Ms. Greenway said.
“The Balanced Scorecard we just embraced as a system,” said Katie Reeves, a member of the Fulton County board of education. “I’m from a business background, and it certainly appealed to my business nature.”
It has “allowed our teachers to use the information, and not just as an end-of-the-year, targeted instruction [tool],” Ms. Reeves said. “It’s a management tool that survives different board members, different superintendents.”
Overcoming Fear
Implementing the Balanced Scorecard system was a marked change from the way that performance had previously been measured in the Fulton County district, and it took some time for teachers to adjust, according to Ms. Greenway.
In the past, it was hard to define targets because of the lack of consistency in the available data, she said. Achievement was measured by standardized tests, but not every grade level or subject was tested, so it was difficult to analyze performance, she said.
“People are very anxious if they feel like they’re being held accountable for something when they don’t have the resources and skills,” Ms. Greenway said. But once teachers became comfortable with the system and learned more about how it could be used, they embraced it, she said.
Elizabeth M. Laird, the program manager for the Data Quality Campaign, agrees that a shift to a more data-centered approach can be intimidating for teachers at first.
“Having a fear of data is completely rational, but that cultural change is under way in education,” Ms. Laird said. “[Data] is not just something that’s used to rank schools or teachers, but something that really can help them do their job.”
To help teachers see that connection, she said, district leaders have to be “clear about how the data will be used in a way that supports improving student achievement.”
The passage of the federal No Child Left Behind Act nearly eight years ago sparked a renewed focus on collecting and reporting data, and advancements in technology have dovetailed with that emphasis, said Jane A. Lockett, a retired IBM consultant specializing in the education industry and business intelligence.
“Now we’re starting to implement ... intelligence systems that are able to harvest data, combine it, or integrate it, and that really gives a very personal and customized view of how a teacher’s students look,” she said. “We’re finally getting to the point where systems are being used that help customize and personalize teaching and learning.”
Going Beyond Hunches
As a result, Ms. Lockett said, school leaders should now be asking themselves: “What is the true data we need to collect? What are the questions we’re trying to ask and the answers we hope to find?”
Better use of data can also reduce subjectivity and interpersonal friction in decisionmaking, suggested Diane Kline, the senior project manager for APQC Education.
“When you have the data in front of you, the discussion becomes about the data versus about the people, and that does a number of things,” she said. “One is, it kind of cuts down on any sort of defensive or interpersonal issues that you might run into, and it makes the discussions so much more valid because it’s not based on opinions or hunches.”
In addition to using Fulton County’s data-management system to bolster student achievement, district leaders have tapped the system to improve their operations, said Ms. Greenway.
The amount of data collected about the demographics and other characteristics of Fulton County schools allows officials to compare the district with other, similar school systems. For example, the data-management system allows them to compare their student-transportation system with that of a similarly sized district to see if there might be a better, more efficient setup.
“We don’t have time, in education, to re-create the wheel,” said Ms. McGraw, the executive director of continuous improvement. “You’ve got to be willing to say: ‘We’re not perfect. Maybe there’s a better way to do it.’“
Despite a high turnover in superintendents in recent years—the district went through five between 2000 and 2005—the use of data has remained a high priority for the school board, said Superintendent Cindy Loe, who has been in that position about a year.
“What really sets the district apart from others that I’ve seen that have had a lot of changes in superintendent leadership is that the board is very stable,” she said.
And when choosing a new superintendent, the board has purposely looked for someone who has a commitment to data and continuous improvement, where the data collected is constantly evaluated and used to adjust instruction, as well as operational processes, she said.
That emphasis has led to “a culture [of data in Fulton County],” said Katie Lovett, the project director for the Data-Driven Decision Making Initiative at the Consortium for School Networking, or COSN, and a former chief information officer of the Fulton County schools. “The culture does not happen overnight, and it does not happen without strong leadership.
“If the school district doesn’t make the commitment, not only to the technology, but for using the data” in a day-to-day way, she said, the data system won’t be effective.
Superintendent Loe said she has also brought an emphasis on “value-added analysis” to the district. “I would like to enhance the data-management tools by including a focus on value-added analysis where we are really trying to determine what value each school or department is adding to the students,” she said. Determining that value, she said, requires a more longitudinal look at the data, rather than a snapshot.
‘Deliberate and Logical’
Ms. Loe has also pushed for cross-divisional “action teams,” which gather members from different departments to utilize data to tackle a particular issue, such as figuring out what textbooks to use, maximizing gifted and talented programs, or examining the district’s health insurance.
Such teams “break down the bureaucracy that can happen in a large organization,” she said. “Every voice is heard.”
In addition, the teams are asked to make recommendations on their respective topics within 90 days—a requirement designed to help move those suggestions forward.
“There has been this common vision that we are building a data-driven organization that’s focused on continuous process improvement, and these are all the building blocks we have to put in place” to achieve that goal, said Deputy Superintendent Greenway. “It’s just been very deliberate and logical. One step after the other.”
