🔗 Share this article

Technology companies and child safety agencies will receive authority to evaluate whether artificial intelligence systems can generate child exploitation images under recently introduced UK legislation.

Significant Rise in AI-Generated Harmful Content

The declaration coincided with revelations from a protection watchdog showing that cases of AI-generated CSAM have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025.

Updated Regulatory Framework

Under the amendments, the government will permit approved AI companies and child safety organizations to examine AI models – the foundational systems for conversational AI and image generators – and verify they have adequate protective measures to stop them from producing images of child exploitation.

"Ultimately about preventing abuse before it occurs," declared Kanishka Narayan, adding: "Specialists, under rigorous protocols, can now identify the risk in AI models early."

Tackling Regulatory Challenges

The amendments have been introduced because it is against the law to create and possess CSAM, meaning that AI creators and other parties cannot generate such content as part of a evaluation regime. Until now, authorities had to delay action until AI-generated CSAM was published online before addressing it.

This law is designed to preventing that problem by enabling to stop the creation of those materials at their origin.

Legal Framework

The changes are being added by the authorities as revisions to the crime and policing bill, which is also implementing a ban on owning, creating or sharing AI systems developed to create exploitative content.

Practical Impact

This recently, the official toured the London base of a children's helpline and listened to a simulated conversation to counsellors featuring a report of AI-based exploitation. The call portrayed a teenager requesting help after being blackmailed using a explicit AI-generated image of themselves, created using AI.

"When I hear about children facing blackmail online, it is a source of intense frustration in me and justified concern amongst families," he said.

Concerning Statistics

A prominent online safety organization stated that cases of AI-generated exploitation content – such as online pages that may include multiple images – had more than doubled so far this year.

Instances of category A content – the gravest form of exploitation – rose from 2,621 visual files to 3,086.

• Girls were predominantly targeted, making up 94% of illegal AI depictions in 2025
• Depictions of infants to two-year-olds rose from five in 2024 to 92 in 2025

Industry Response

The legislative amendment could "constitute a crucial step to ensure AI products are secure before they are launched," stated the chief executive of the internet monitoring foundation.

"AI tools have enabled so survivors can be targeted all over again with just a few clicks, giving criminals the capability to make potentially limitless quantities of sophisticated, lifelike exploitative content," she continued. "Content which additionally exploits survivors' suffering, and makes young people, particularly girls, more vulnerable on and off line."

Counseling Interaction Information

The children's helpline also published information of support interactions where AI has been mentioned. AI-related harms discussed in the conversations comprise:

• Using AI to rate body size, physique and appearance
• Chatbots dissuading young people from talking to safe adults about harm
• Being bullied online with AI-generated content
• Digital extortion using AI-manipulated images

During April and September this year, the helpline delivered 367 support sessions where AI, chatbots and related terms were mentioned, four times as many as in the equivalent timeframe last year.

Half of the references of AI in the 2025 interactions were connected with psychological wellbeing and wellness, encompassing using AI assistants for support and AI therapy applications.