The social media platform X has provoked major anger after users employed its AI chatbot Grok to alter pictures of women by taking off their garments.
Multiple examples have emerged where the technology has been used to remove clothing from female subjects to show them in swimwear non-consensually, as well as to insert them in sexualized situations.
Company Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to ban nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "face a jail term and heavy fines".
"X or Grok could prevent these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their networks, but did not specify whether it was at this time looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional features—which responds to X users' queries when they tag it in a post.
It is frequently used to provide reactions or additional context to other posters' comments, but people on X are also capable of alter an uploaded photo through its AI image editing tool.
History of Controversy and Rules
The tool has been criticised for enabling users to create pictures and footage with nudity and sexually suggestive content, and it was previously alleged to have been involved in generating sexually explicit content of celebrities.
"X has been enabling the generation and sharing of these pictures for months without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "minimize the danger" of users encountering unlawful material on their services and to take down it promptly when they learn of it.
