The recent disclosure by OpenAI of a security incident involving Hugging Face has offered a practical, real-world illustration of what can happen when an autonomous system is given a goal, access to tools, and enough authority to pursue that goal across technical boundaries, according to Nicolas Blank, CTO of NBConsult, a Cyberlogic company.
“According to OpenAI’s preliminary account, models being evaluated for advanced cyber capability found a way through a constrained package proxy via a zero-day exploit, obtained internet access, escalated privileges, and used stolen credentials and further vulnerabilities to reach Hugging Face’s production systems in search of benchmark answers,” says Blank. Hugging Face detected and contained the activity, and the investigation is ongoing.
While the model exploited a zero-day finding in the sandbox, Blank argues this is just as much an identity and access incident as it is a technical one. “The models acted through the systems, identities and permissions available to them,” he says. “Their capability affected how they pursued the objective; identity and access controls determined where they could go and what they could do once they arrived.”
OpenAI has attributed the breach to a zero-day exploit, but Blank believes that does not reduce the accountability organisations carry for how their own systems are governed.
