CYBER-CRIME
Chinese open-weight model GLM 5.2 happily obliged
Apparently, being a leading destination for AI development doesn't mean AI will bail you out. AI agents broke into Hugging Face's production infrastructure, but commercial LLM guardrails blocked the forensic investigation, forcing it to turn to a Chinese open-weight model instead.The intrusion, “driven, end to end, by an autonomous AI agent system,” compromised a “limited set” of Hugging Face’s internal datasets and “several” credentials used by its services, according to a Thursday security incident disclosure.While the ML platform says that it’s still investigating whether any partner or customer data was exposed in the breach, there’s “no evidence of tampering with public, user-facing models, datasets, or Spaces, and our software supply chain (container images and published packages) was verified clean.”
It also doesn't know which model the attackers used to power a swarm of AI agents, which, we're told, executed many thousands of individual actions across short-lived sandboxes, using self-migrating command-and-control staged on public services.
“This matches the ‘agentic attacker’ scenario the industry has been forecasting,” according to the Hugging Face blog.Additionally, after unsuccessfully using unnamed frontier models to start the forensic analysis, the Hugging Face security team ultimately ran the log analysis on GLM 5.2, an open-weight model developed by Chinese AI firm Z.ai, on the platform’s own infrastructure.The advanced commercial models didn’t work because their analysis required submitting real attack commands, exploit payloads, and command-and-control artifacts - all of the things that the LLMs’ guardrails have been trained to block so that the AI systems can’t be used in real-life attacks.“The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried,” the security team wrote, noting that it’s not arguing against safety measures on hosted models and has shared this information with the LLM providers.Using GLM 5.2 had another benefit, Hugging Face noted: “No attacker data, and none of the credentials it referenced, left our environment.”This also serves as an important reminder to defenders, according to the AI platform. “Have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.”
