“A company whose users request just ten images in violation of the statute would face exposure up to $5 million in civil penalties alone. A company with a thousand violative images could be fined up to $500 million. And a business whose users created a hundred thousand images covered by [the law] (not at all unlikely for a publicly available program with millions of users generating billions of images) could owe an eye-popping $50 billion dollars.”
Additionally, the law gives victims a right to sue xAI over any individual output, which increases xAI’s financial risks.
The penalties are so severe, xAI said in its lawsuit that it was finally preparing to update Grok to block harmful outputs after more than six months of backlash and probes pressuring the firm to tighten its safeguards.
“Confronted with $500,000-per-image strict liability and no safe harbor, xAI has no practical choice but to restrict Grok Imagine’s image-editing features in various ways when the statute takes effect on August 1, 2026,” xAI argued. “Protected speech freely available before the law takes effect will thus be chilled.”
However, xAI would prefer to leave Grok unchanged and continue relying on its terms of use stipulating that users could be banned for using Grok to make CSAM or other kinds of non-consensual intimate images (NCII), its complaint said.
