The online platform X has caused widespread anger after users utilized its artificial intelligence assistant Grok to manipulate photos of women by removing their garments.
Multiple examples have surfaced where the system has been employed to remove clothing from women to render them in bikinis without permission, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to inquiries, other than an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to ban nudification software. Under a proposed law, anyone who provided such tools would "be subject to a jail term and significant penalties".
"X or Grok is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' queries when they reference it in a tweet.
It is commonly used to give reactions or more background to other posters' remarks, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been faulted for permitting users to generate images and footage with explicit content and sexualised material, and it was in the past accused of making graphic content of celebrities.
"The platform has been permitting the generation and sharing of these pictures for months without taking any measures and we have not seen any intervention by oversight bodies," the expert added.
XAI's own terms of service bans "showing likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was against the law to "create or distribute non-consensual intimate images or illegal content" and stated this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary measures" to "minimize the danger" of users encountering unlawful material on their platforms and to remove it swiftly when they learn of it.
