AI Deepfakes Endanger Children Experts Warn

The rise of sophisticated AI-generated images depicting child abuse is creating a critical challenge for child protection agencies, potentially diverting resources from real victims, warns the Internet Watch Foundation (IWF).

AI-generated images of children are on the increase and are causing concern, experts say

The IWF, an organization based near Cambridge, UK, is at the forefront of identifying, flagging, and removing images and videos of child sexual abuse from the internet. However, they've seen a dramatic 300% increase in AI-generated abusive images in 2024 compared to the previous year, adding significant complexity to their mission.

The Blurring Lines Between Real and AI Generated Abuse

Dan Sexton, IWF's chief technology officer, expressed deep concern over this development. He explained that law enforcement and other agencies now face a perilous situation: they could be "trying to rescue children that don't exist or not trying to rescue children because they think they're AI."

"About two years ago we first started seeing this content being circulated, and there were little 'tells' - it looked distinctly different," Mr Sexton said. But advancements in AI technology mean that distinguishing these fakes from reality has become incredibly difficult, at times impossible.

"There will be imagery in there that is so realistic or so similar to the content we see, you cannot tell the difference," Mr Sexton added. He emphasized the core danger: "The risk there, is, we and policing end up trying to rescue children that don't exist or not trying to rescue children because they think they're AI."

Dan Sexton said he was concerned chasing AI-generated children could put real victims at risk

A Soaring Threat The Rapid Rise of AI Content

Mr Sexton highlighted the challenge of tracking the origin and distribution of AI-generated content, especially when it becomes intermingled with genuine abuse material. "When it starts to get shared with real content as well, there's every chance we won't know the AI-generated content is out there," he said. This directly impacts "the ability to safeguard children, and the risk that there would be children who won't get safeguarded because we're too busy dealing with synthetic children."

Last year, the IWF documented a 300% surge in AI-generated abusive content compared to the year before. This explosion in volume underscores the urgency of the situation.

The Human Cost Misdirected Efforts and Revictimization

Natalia (not her real name), an analyst at the IWF for nearly five years with a specialization in AI, corroborated these concerns. She described the task of distinguishing AI from real child imagery as "more and more difficult."

Natalia said early signs of AI were simpler to detect as limbs, digits and clothing texture were often "giveaways"

"The content has become so realistic and also the speed at which this technology is developing is really alarming," Natalia stated. She noted that while the IWF first saw AI images in 2023, the number of reports quadrupled in 2024. Like Sexton, she fears that law enforcement might be "sent chasing a non-existent child."

"If we think a child is in danger now, we will make a referral [to the police] and we really don't want to make a referral about an AI-generated child," she explained.

Natalia also shared a disturbing example of how AI is used to compound the trauma of real victims. She spoke of a child abuse victim whose actual images had been circulating since 2011. Even after the abuser was caught and the victim shared her story publicly, the nightmare continued. "Now we are seeing new images of her - images generated by AI - some of them are even more severe than the images that were actually taken in reality," Natalia revealed. "This is as far from a victimless crime as it gets - there's a very real victim here and I think real harm is being done by this content."

Fighting Fire with Fire Using AI to Combat AI

The IWF is not standing idly by. The foundation is actively exploring the use of AI to detect AI-generated material. "The scale of the problem - and the potential increase in the scale... means it's never been more important to have AI tools.. to help us," Mr Sexton affirmed.

Despite these efforts, the challenge is immense. "I'd like to one day be able to show a report that says there's less [child sexual abuse imagery] but unfortunately that's not the case - it's not happened so far," Sexton lamented.

The Internet Watch Foundation's offices are in a village near Cambridge

National Agencies Grapple with the Evolving Threat

The National Crime Agency (NCA) acknowledged the gravity of the situation. A spokesperson stated, "Generative AI image creation tools will increase the volume of child sexual abuse material available across the clear web and dark web, creating difficulties with identifying and safeguarding victims due to vastly improved photo realism."

The NCA affirmed its commitment to tackling this threat: "However, we are working closely with partners to tackle this threat, and are continuing to invest in technology to assist us with CSA (child sexual abuse) investigations to safeguard children."

The proliferation of AI-generated child abuse content presents a formidable and evolving challenge, demanding urgent and innovative responses from tech companies, law enforcement, and child protection organizations worldwide.