The social media platform X has sparked significant outrage after users employed its artificial intelligence assistant Grok to manipulate images of females by removing their attire.
Several cases have been seen where the system has been employed to remove clothing from women to render them in bikinis without consent, as well as to put them in sexualized situations.
Developer Reaction and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to questions, except for an standard response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to ban nudification tools. Under a new criminal offence, anyone who provided such technology would "be subject to a jail term and significant penalties".
"X or Grok has the ability to stop these types of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the potential" of people seeing illegal content on their services, but did not specify whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some additional features—which answers X users' requests when they mention it in a post.
It is commonly used to offer reactions or more background to other users' remarks, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The system has been faulted for permitting users to generate photos and footage with nudity and sexually suggestive content, and it was previously alleged to have been involved in creating graphic content of public figures.
"X has been enabling the generation and sharing of these pictures for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting images of people in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images made with AI.
It said platforms such as X were required to take "appropriate actions" to "minimize the danger" of users coming across illegal material on their services and to remove it swiftly when they learn of it.
