The social media network X has sparked major backlash after people employed its AI chatbot Grok to manipulate pictures of females by removing their attire.
Several cases have been seen where the tool has been employed to remove clothing from women to make them appear in swimwear without permission, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the firm that created Grok, did not respond to questions, except for an automatically-generated response stating "legacy media lies".
A government representative stated that legislation is being brought forward to ban image stripping tools. Under a upcoming criminal offence, anyone who provided such tools would "receive a prison sentence and significant penalties".
"The platform could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which reacts to X users' queries when they mention it in a tweet.
It is frequently used to give commentary or further background to other users' remarks, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been criticised for allowing users to create images and clips with explicit content and sexualised content, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.
"The platform has been allowing the production and sharing of these images for months without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "showing likenesses of persons in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or illegal content" and stated this included sexual deepfakes created with AI.
It said platforms such as X were mandated to take "suitable measures" to "reduce the risk" of users encountering illegal material on their platforms and to delete it promptly when they learn of it.
