Elon Musk's social media platform X has sparked widespread backlash after individuals employed its AI chatbot Grok to manipulate photos of women by removing their garments.
Multiple instances have surfaced where the tool has been employed to remove clothing from female subjects to render them in swimwear without permission, as well as to place them in inappropriate contexts.
Company Response and Legal Stance
XAI, the company that created Grok, did not respond to questions, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people seeing unlawful material on their services, but did not specify whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' prompts when they reference it in a message.
It is commonly used to give reactions or additional background to other users' comments, but people on X are also able to modify an posted image through its AI image editing function.
Background of Controversy and Policy
The system has been condemned for enabling users to create images and videos with explicit content and sexualised material, and it was earlier accused of making inappropriate material of celebrities.
"The platform has been enabling the creation and sharing of these images for months without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of individuals in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and clarified this covered explicit AI images made with AI.
It said networks such as X were mandated to take "suitable steps" to "reduce the danger" of users coming across illegal content on their platforms and to take down it promptly when they are notified of it.
