🔗 Share this article

Technology companies and child safety agencies will be granted authority to evaluate whether artificial intelligence tools can produce child abuse material under recently introduced UK laws.

Substantial Increase in AI-Generated Illegal Content

The announcement came as revelations from a safety monitoring body showing that reports of AI-generated CSAM have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

Updated Regulatory Framework

Under the amendments, the government will permit designated AI companies and child safety groups to examine AI models – the foundational technology for chatbots and visual AI tools – and verify they have sufficient safeguards to stop them from creating images of child exploitation.

"Fundamentally about preventing exploitation before it occurs," stated the minister for AI and online safety, noting: "Specialists, under strict protocols, can now identify the risk in AI models promptly."

Addressing Legal Obstacles

The changes have been implemented because it is illegal to produce and own CSAM, meaning that AI creators and other parties cannot create such images as part of a testing process. Until now, authorities had to delay action until AI-generated CSAM was uploaded online before addressing it.

This legislation is designed to preventing that issue by enabling to halt the creation of those images at source.

Legislative Structure

The changes are being introduced by the government as modifications to the criminal justice legislation, which is also establishing a prohibition on owning, creating or distributing AI models developed to create exploitative content.

Real-World Consequences

This week, the minister toured the London base of Childline and heard a simulated call to counsellors involving a account of AI-based exploitation. The call portrayed a teenager requesting help after facing extortion using a sexualised AI-generated image of himself, constructed using AI.

"When I learn about young people experiencing extortion online, it is a source of extreme anger in me and justified anger amongst families," he said.

Alarming Statistics

A leading online safety organization stated that instances of AI-generated abuse material – such as online pages that may contain multiple files – had significantly increased so far this year.

Cases of the most severe material – the gravest form of exploitation – rose from 2,621 visual files to 3,086.

• Girls were predominantly targeted, accounting for 94% of illegal AI images in 2025
• Portrayals of newborns to toddlers rose from five in 2024 to 92 in 2025

Sector Response

The law change could "constitute a vital step to guarantee AI tools are secure before they are released," commented the chief executive of the online safety foundation.

"AI tools have made it so survivors can be targeted repeatedly with just a few clicks, giving offenders the ability to make possibly limitless quantities of advanced, photorealistic exploitative content," she continued. "Material which further exploits victims' trauma, and renders young people, especially girls, less safe both online and offline."

Support Session Data

Childline also released information of support interactions where AI has been mentioned. AI-related risks discussed in the sessions include:

• Using AI to rate body size, body and appearance
• Chatbots discouraging children from consulting safe adults about harm
• Being bullied online with AI-generated content
• Online blackmail using AI-manipulated pictures

During April and September this year, the helpline delivered 367 support sessions where AI, chatbots and related topics were mentioned, significantly more as many as in the same period last year.

Fifty percent of the references of AI in the 2025 interactions were related to psychological wellbeing and wellbeing, encompassing utilizing chatbots for support and AI therapeutic apps.