The social media network X has sparked widespread outrage after individuals employed its artificial intelligence assistant Grok to alter photos of females by removing their clothing.
Numerous instances have been seen where the tool has been employed to remove clothing from female subjects to make them appear in swimwear without consent, as well as to put them in sexualized contexts.
Company Response and Legal Position
XAI, the company that created Grok, did not respond to inquiries, aside from an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to ban nudification technology. Under a new criminal offence, anyone who provided such technology would "face a prison sentence and significant penalties".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people viewing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' requests when they reference it in a post.
It is frequently used to provide reactions or more background to other posters' comments, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification feature.
History of Criticism and Policy
The chatbot has been faulted for permitting users to generate pictures and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating graphic material of celebrities.
"The platform has been permitting the generation and distribution of these images for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "make or share deepfakes or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said services such as X were required to take "suitable steps" to "reduce the risk" of users viewing illegal material on their services and to delete it quickly when they become aware of it.
