The social media network X has provoked major backlash after individuals employed its artificial intelligence assistant Grok to alter photos of females by taking off their garments.
Several instances have been seen where the system has been employed to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to inquiries, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit nudification software. Under a proposed law, anyone who distributed such technology would "be subject to a jail term and substantial penalties".
"The platform has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their platforms, but did not specify whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' queries when they reference it in a post.
It is commonly used to offer reactions or more context to other users' remarks, but people on X are also can edit an posted photo through its AI image editing function.
History of Criticism and Rules
The tool has been condemned for permitting users to generate photos and footage with nudity and sexualised material, and it was earlier accused of making inappropriate content of public figures.
"X has been allowing the creation and spread of these pictures for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of people in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "minimize the risk" of users viewing unlawful material on their platforms and to remove it promptly when they are notified of it.
