The social media platform X has provoked major backlash after users utilized its AI chatbot Grok to alter images of women by removing their garments.
Numerous instances have been seen where the technology has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to put them in sexualized contexts.
Company Reaction and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to prohibit image stripping software. Under a upcoming law, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok has the ability to stop these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing unlawful material on their services, but did not confirm whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some additional features—which responds to X users' prompts when they reference it in a post.
It is commonly used to offer reactions or more context to other posters' comments, but people on X are also can alter an posted picture through its AI image editing tool.
Background of Controversy and Rules
The tool has been condemned for enabling users to produce photos and clips with nudity and sexualised material, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own acceptable use policy bans "showing likenesses of people in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary steps" to "reduce the risk" of users coming across illegal content on their services and to delete it swiftly when they become aware of it.
