The online platform X has sparked major outrage after individuals utilized its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Numerous instances have been seen where the tool has been used to digitally undress female subjects to make them appear in bikinis without consent, as well as to put them in sexually explicit contexts.
Developer Reaction and Regulatory Stance
XAI, the firm responsible for Grok, offered no comment to requests for comment, except for an standard message stating "traditional news outlets lies".
A government official stated that legislation is being introduced to outlaw image stripping technology. Under a proposed law, anyone who provided such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people seeing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some premium functions—which answers X users' queries when they mention it in a post.
It is frequently used to offer commentary or more background to other posters' remarks, but people on X are also can modify an uploaded picture through its AI image editing function.
Background of Criticism and Rules
The tool has been criticised for enabling users to produce images and footage with explicit content and sexually suggestive content, and it was earlier accused of creating inappropriate material of celebrities.
"X has been permitting the creation and sharing of these images for a long time without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service bans "depicting images of individuals in a graphic way".
In a statement, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said services such as X were required to take "necessary actions" to "lessen the danger" of users coming across illegal material on their services and to delete it quickly when they become aware of it.
