The story so far: Five anonymous individuals — including at least one minor — filed a proposed class action lawsuit in July against Elon Musk’s xAI, and Stability AI, over the alleged use of the companies’ online features to make child abuse material. The lawsuit pertains to X platform’s Grok chatbot that was capable of creating sexual content using real people’s photos. The lawsuit also alleged that Stability AI tried to grow its business by enabling its AI models to create sexually explicit content.
How to refer to content that contains child abuse
CSAM: This acronym stands for Child Sexual Abuse Material. An alternative form—CSEM, standing for Child Sexual Exploitation Material—is also used in some contexts.
CSAM/CSEM is used in place of the phrase “child pornography,” thereby indicating that no such legal category can ever exist, as all forms of “child pornography” are inherently criminal and abusive.
What is the case about?The proposed class action lawsuit has been brought forward by five female individuals in the U.S., against xAI and Stability AI. In the lawsuit, they are identified as Jane Doe 1-5.At least one of them was a minor as of July 7, while the others were minors during the relevant time period. The legal filing explained how the five discovered that pictures of them as children had been morphed with AI in order to create child abuse material.The lawsuit aims to cover not just the affected individuals, but all people in the United States who had real images of themselves as minors altered by xAI/Grok or a Stability AI-powered app to produce sexual deepfakes. The five are seeking civil remedies for personal injuries suffered as a result of the companies’ alleged conduct.Originally filed on March 16 this year, the suit was later expanded in a July 7 filing to include the allegations of two more anonymous female individuals. Stability AI was also added as a defendant at the later stage.Which feature in Grok and Stability AI enabled the creation of morphed images?Users can now upload real photos into advanced AI apps in order to edit them into new images or even short videos, with just a few prompts. While safety filters should stop users from morphing photos of real children or creating sexually abusive content, this is not always the case. Sex predators often scrape photos from public social media accounts in order to morph them with AI.Coming to the lawsuit, the cases of the first three anonymous individuals are linked, as they were part of the same criminal investigation. With reference to the first victim, the perpetrator allegedly used both Stability AI as well as Grok to edit regular childhood photos into morphed, sexual images.Regarding the second and third victims, the perpetrator allegedly used a phone-based app that “relied on Stability AI’s image-producing tools” to create sexual deepfakes of children.The fourth victim was allegedly targeted by their stepfather, who is said to have used Grok to create sexual deepfakes out of an existing, non-explicit image. In the case of the fifth victim, the perpetrator allegedly used the Grok application on a phone to create the deepfakes.What are the complainants saying about safeguards in xAI and Stability AI?The plaintiffs stress that xAI and Stability AI did not incorporate safeguards to prevent the production of child abuse material, and that the two companies also saw the profit potential of sexual content generated by AI.“Where others saw a risk that required mitigation, xAI—and its founder Elon Musk—saw an opportunity to profit off the sexual predation of real people, including children,” stated the filing.“Stability AI learned that nothing hooked users like releasing an AI model that freely produced “NSFW” content, including of children. Lax and inadequate safety restrictions were an easy way to produce exponential user growth of their models, no matter the harm this might cause a generation of kids,” it added.In the case of the fourth victim, their stepfather had allegedly prompted Grok to generate approximately 7,000 sexually explicit images/videos depicting the person, based on their real, non-explicit childhood photo. After a rape-related prompt reportedly triggered xAI to escalate the incident, the legal filing claimed that Elon Musk’s company “obstructed” the police investigation at “every turn,” which resulted in delays when tracking down the maker of the abusive images.After he was arrested, charged with child exploitation offences, and released on bail, the stepfather died by suicide in March, turning the victim’s life into a “nightmare,” per the lawsuit. The filing stated that without the “ease” and “access” that AI like Grok offered, the perpetrator would not have been able to make child abuse material depicting the stepdaughter.The legal filing detailed that the five individuals experienced serious emotional distress as a result of the AI child abuse material. Some of them were struggling with school or feeling unable to live normal lives due to their fear, while others worried that the AI images could even lead to offline stalking as more child sex predators accessed the images online.“Their lives have been shattered by the devastating loss of privacy, dignity, and personal safety that the production and dissemination of this CSAM has caused and will continue to cause,” stated the filing.What was the companies’ response?While a formal response to the amended complaint is yet to come, Mr. Musk had earlier refused to accept that Grok could create sexual deepfakes of children.“I [am] not aware of any naked underage images generated by Grok. Literally zero. Obviously, Grok does not spontaneously generate images, it does so only according to user requests,” he posted on X on January 14, going on to add that Grok would refuse to produce illegal images and that bugs related to adversarial hacking would be fixed “immediately.”In February, Stability AI announced that it had joined the Tech Coalition, and become a member of the group of technology companies committed to protecting children from online sexual abuse.Elon Musk’s X, where AI-generated sexual deepfakes spread unchecked across the platform this year, was also listed as a member.(Those in distress or having suicidal thoughts are encouraged to seek help and counselling by calling the helpline numbers here)
