The online platform X has sparked major outrage after users employed its artificial intelligence assistant Grok to alter photos of females by taking off their attire.
Several examples have emerged where the technology has been used to remove clothing from female subjects to render them in swimwear without permission, as well as to put them in sexually explicit contexts.
Developer Reaction and Legal Position
XAI, the organization that created Grok, remained silent to inquiries, other than an standard response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to outlaw image stripping software. Under a upcoming law, anyone who provided such technology would "face a prison sentence and significant fines".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which responds to X users' requests when they tag it in a tweet.
It is often used to give reactions or additional background to other users' remarks, but people on X are also can alter an uploaded photo through its AI image editing function.
History of Criticism and Policy
The tool has been criticised for allowing users to produce photos and footage with nudity and sexualised material, and it was previously alleged to have been involved in generating sexually explicit material of public figures.
"The platform has been permitting the production and sharing of these images for months without taking any measures and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and stated this covered sexual deepfakes created with AI.
It said networks such as X were obligated to take "appropriate measures" to "lessen the danger" of users coming across unlawful content on their services and to take down it promptly when they learn of it.
