School district tech leader William Brackett uses a compelling example to illustrate how quickly artificial intelligence is changing the cat-and-mouse game in cybersecurity.
Brackett, the director of technology for the Oak Park Elementary school district near Chicago, said he knows the head of a small cybersecurity company that does cyber penetration testing, also known as “pen testing,” for school districts, mimicking actual cyberattacks to identify weaknesses that could be exploited by real hackers. The head of that company used to employ a team of seven people to do the pen testing—he now does it completely by himself: AI can create up to five pen testing programs that can work relentlessly 24 hours a day.
But, the problem is, a student with bad intentions, or almost anyone, can now also deploy multiple AI models, essentially creating their own hacking crew, all driven by one person.
“AI on the attack side can be devastating,” said Brackett, whose 5,500-student district provides digital learning devices for its K-8 students.
Brackett recently spoke with Education Week about a host of cybersecurity challenges facing schools as the use of AI expands in nearly all sectors of society. He explained the vital role of training and education, the most vulnerable spots in school districts, the rising use of deepfake audio and video, and how hackers’ email phishing attempts are becoming more sophisticated.
This conversation has been edited for length and clarity.
What is one of your biggest concerns about how AI is affecting cybersecurity?
On the audiovisual side, AI is starting to get to the point where we can’t trust what we normally have as people for a very long time, and that is trust our voices, trust our eyesight.
Think about how many superintendents have their voices recorded because we record our board meetings and reports and speeches and all this other stuff. Now, [the hacker] can set something up in AI and have a [fake] superintendent call and leave a message: “I need this purchasing order done now.” And now we have that person in finance use their contextual authentication that they’re used to [i.e. the superintendent’s voice], and they say “oh, that is the superintendent, I talk to her every day.” And then all of a sudden, there’s a breach.
What else worries you?
AI mimicking writing styles. When you have folks who are prolific at writing, then now you have a writing style that you can mimic and ask an AI model: “Hey, take this writing style, this person, and generate this email.” Those are things that scare me and keep me up at night.
You also mentioned AI being a manpower magnifier or a research assistant. What does that look like?
[In the past], it might take a hacker a couple weeks to build a profile of an attack surface for a school district. Now, a hacker can put your school district in Gemini, and just say, “what are their top technology vendors? What’s their security plan?” The AI will go out and pull everything from public sources and provide an attacker a condensed version in seconds.
How can schools protect themselves?
We educate our financial and human resources people first. Those are our big ones. That’s where the money comes out. That’s where the personally identifiable information sits.
Next, our student information system folks. And then we go to our staff.
Most schools are going to have to start looking at higher-end training and also auditing. You might want to test employees every so often just to make sure that they’re compliant with what they’re doing. And, then, absolutely reward them when they do well and train them when they fail.
How can AI tools help you defend against potential attacks?
What AI is going to help me do is filter out [the data noise] and start finding patterns because AIs are very much like humans—they find patterns. When they find patterns that do not match the normal patterns of what my students do, they can flag them. These tools will allow us to add some automation to the process, probably within the next year.
Any potential downsides to using AI for cyber defense?
You can get hallucinations [cybersecurity parlance for inaccurate information].
I also probably wouldn’t turn wholehearted automation on because if it’s not configured properly and if hackers know you are running it, they could try to feed it bad information to start trying to shut down everything.
Data analysis for this article was provided by the EdWeek Research Center. Learn more about the center’s work.
