The New York State Education Department has put the brakes on a school district’s planned trial run of new facial recognition software aimed at improving school safety, after civil liberties advocates raised alarm bells about the program’s impact on student privacy.
The Lockport City school district, located near the Canadian border, had been scheduled to go through a planned test of the new system next week. But the state education department wants the district to more carefully evaluate the system to be sure that “student data will be protected with the addition of the new technology,”
“The department is in contact with the district. We have made it clear, the Department has not approved the testing of the system planned for next week and we told the District not to commence the testing of the technology until we receive information that assures us that student information will be properly protected,” a state department spokesman said in an email. The Lockport Union-Sun&Journal first reported the news.
If the state hadn’t hit the pause button, the Lockport City School District would have become one of the first school systems in the country to try out facial recognition software. The district had planned to use Aegis software, created by a Canadian-based company, to alert district officials if someone on a flagged list of individuals showed up at one of the district’s eight schools. The district had planned a test-run of that system next week, and hoped to have it fully in place by next school year.
The software can also detect ten different types of guns, according to the Union-Sun&Journal. If the system spotted one, law enforcement would be alerted.
Who might be flagged under the system? Students who have been suspended, staff suspended or on administrative leave, certain sex offenders, anyone prohibited from entering district property or anyone believed to pose a threat, according to the Union-Sun&Journal.
The district covered the cost of the software using $1.4 million of a $4.2 million state grant.
Lockport City schools did not respond to requests for comment from Education Week.
Software that can pinpoint individual faces on a surveillance video is highly controversial, even though it’s already been purchased by law-enforcement agencies throughout the country. In fact, it helped officials in Maryland identify the shooter at an Annapolis, Md., newspaper last year.
But facial recognition software has run into big criticisms from civil rights and privacy advocates, who worry about the implications of storing faces in a database, and potential biases in the software itself. In fact, the city of San Francisco—near the heart of Silcon Valley—recently banned the use of the technology.
“Facial recognition tech is notoriously inaccurate,” said Stefanie Coyle, an education counsel for the New York Civil Liberties Union, the state’s American Civil Liberties Union affiliate, in an interview with Education Week. In particular, she said it has trouble identifying people of color, women, and children.
And then there are the privacy concerns, particularly for children who can’t opt-out of the new system and are required by law to attend schools.
“Obviously, a concern with any sort of tech like this is vulnerabilities to hacking or cyber security,” she said. If your credit card is hacked, you could ask for new one but “you can’t get a new face,” Coyle said. “We’re talking about [the things] most sensitive and personal about you, collected and stored in a way that could be vulnerable.”
She’s also worried that student facial data could be handed over to immigration officials. “There are just so many unanswered questions about this sort of tech, it just seems so unwise to deploy it on kids.”
And she has concerns about district official’s plans to use the system to support its discipline policies, by flagging kids who are suspended.
The district’s superintendent, Michelle Bradley, told the Union Sun&Journal that she understands that the system has detractors. But, she said, it’s an allowable use of state school safety funding.
“I would say for the Lockport City School District, while it’s controversial, it’s not prohibited and the most important thing is we believe we’ve established boundaries in the use of this,” Bradley told the paper. “We have a policy that intends to protect privacy. We have identified a small group of individuals who will be placed in a database.”
Image: Getty Images
A version of this news article first appeared in the Digital Education blog.