Keeping children safe from the potential harms of artificial intelligence isn’t just a problem for educators and policymakers here in the United States.
It’s something the international community—including the United Nations—is exploring.
Back in July, the organization launched the Coalition for Children’s Rights and Protection in the Age of Artificial Intelligence—a collection of governments, tech companies, child welfare organizations, and others—to consider the issue. (So far, at least 17 countries have signed on, but not the United States.)
As it ramps up these efforts, the U.N. on Sept. 22 convened a meeting involving international organizations, tech companies, experts, and government leaders as part of the U.N. General Assembly this month in New York. The national leaders participating included Prime Minister Anthony Albanese of Australia, President Emmanuel Macron of France, President William Ruto of Kenya, and Prime Minister Pedro Sánchez of Spain.
Macron kicked off the meeting, warning that the international community should move quickly to protect children from the risks of AI—something he said nations around the globe failed to do when social media came on the scene more than a decade ago. (Some nations have acted to regulate social media, including Macron’s France.)
“AI is here and it’s already part of the life of our children,” Macron said. “It is impacting their development.”
The organization also heard from Jim Steyer, the CEO of Common Sense Media, an organization that studies youth and technology.
Steyer pointed to a recently passed California law, which requires chatbots to show that they meet stringent safety standards, as a possible model for internationally backed regulation.
Chris Lehane, the global affairs officer at OpenAI, the developer of ChatGPT, also mentioned the Golden State’s legislation.
And he noted that ChatGPT includes a “study” mode, set up for college students to use to conduct research without cheating, and a teen version.
Also speaking: Erin Mote, the CEO of InnovateEdu, a national nonprofit that works on education policy and technology.
Education Week caught up with Mote on Zoom a few days after the U.N. meeting. The following Q&A is a partial transcript of the conversation, edited for clarity and brevity.
What, in your view was the purpose of this meeting?
The intention of this conversation was to find out: Do we have common ground on child safety and AI, regardless of what approach your government is taking, whether you’re in Australia or Kenya? Do we have agreement that we need a global research agenda? And do we have agreement that we need a global safety standard around child safety and AI?
You were in a room with heads of state and experts, asking for help with AI and child safety. What did you suggest?
Safety is not the counter polarity of innovation; it’s the foundation upon which we build. If we compromise this generation’s well-being, we’re not innovating. We’re mortgaging the future.
It’s abundantly clear that the tech companies can’t self-regulate. AI products need to be designed with safety at their core, independently evaluated, and audited.
We cannot have one set of rules in one country and a different set of rules in another place.
We also need a global research agenda. There’s too many questions that we don’t have answers to about AI and kids and education and even AI and adults and work patterns.
We need a global coalition of researchers working at it, which might even include the Frontier Labs [organizations leading AI development such as OpenAI and Anthropic]. It might include philanthropy. It might include the multinationals.
We can’t wait a decade to understand the promise and the peril of this technology.
When you talk about the potential harms of AI for kids, what are you referring to specifically?
First, we have seen a spike in the creation of deepfake AI pornography images of children. This is a crisis. I think we all can agree that that is that is outside our moral red lines. We need to act, and we need to act swiftly.
And there’s existing law and regulation in every place in the world around child sexual adult material, and so we don’t need new laws.
We do need global collaboration, though, because folks are creating these materials as fast as even the tech companies are taking them down. They’re coming up with new methods to remove some of the technical markers to distinguish something real from a fake.
Two, I think we are very clear that there is foreseeable and demonstrable harm from AI companion chatbots. We need to get those out of schools and classrooms.
Third, [there] is this emerging body of research around cognitive offloading, and even more concerning, cognitive surrender. And so, how are we shaping and developing tools that are actually built for education, anchored in the learning sciences?
Large-language models are optimized for sycophancy, not for Socratic thinking. They are not designed for learning. They’re designed to give you the answers. They’re designed to drive deep engagement patterns. They’re designed to praise you 13 times more than a normal human would.
Say I’m a district leader reading this, and I’m concerned. Do I need to wait for Congress—or the U.N.—to act on these safety concerns?
You don’t have to wait for Congress, and you don’t have to wait for the U.N. You do things right now in your district to protect kids and to put safety at the forefront.
If you’re a superintendent or a district leader, have an incident plan around deepfake intimate images.
You should ban AI parasocial companions. The evidence is overwhelmingly clear. If you have that technology in your school district, get it out.
Be really clear about when you can use AI for assignments and when you can’t, right? And part of that is just helping young people think about: Am I giving up the thinking, am I removing the friction that is actually what helps people learn?
I always say you should never do your first draft in AI. And if we were just clear with kids about these problems, that would be so great.
What about banning or placing or a moratorium on AI, as New York City and Los Angeles have?
I do not think AI moratoriums are helpful. I think that’s the mistake we made with social media.
I was an educator [co-founder of the Brooklyn Laboratory charter school] when we made that mistake.
My 6th grade social studies teacher came to me and said, ‘Don’t you think we should be teaching kids social media and digital literacy?’ And I said ‘No, we’re blocking them. We’re banning them.”
And I think a lot of educators made that same call, and I think that was a massive mistake. Parents want schools to teach AI literacy. Parents want schools to give kids exposure to these tools in a safe environment.
So, when you have the policy in place, and you’re really clear about what you’re trying to do, I think you can use AI. Hopefully, you’re putting measures in place to understand if it’s accomplishing something.
If we decide that we’re just going to ban these tools from classrooms, we’re going to find ourselves in a tough place.
