The social media platform X has sparked major backlash after people utilized its AI chatbot Grok to manipulate photos of women by taking off their garments.
Numerous examples have emerged where the system has been used to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, offered no comment to questions, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification software. Under a new law, anyone who distributed such technology would "face a jail term and substantial penalties".
"X or Grok has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "consider the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium features—which answers X users' queries when they mention it in a post.
It is frequently used to offer reactions or additional background to other posters' comments, but people on X are also can modify an uploaded picture through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been criticised for permitting users to create photos and footage with nudity and sexualised material, and it was earlier alleged to have been involved in generating inappropriate material of celebrities.
"The platform has been permitting the generation and sharing of these pictures for a long time without taking any measures and we have not seen any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "depicting likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said services such as X were mandated to take "necessary steps" to "reduce the risk" of users viewing unlawful material on their platforms and to remove it swiftly when they are notified of it.
