Hugging Face is the online warehouse of artificial intelligence. Anyone can download a working AI model from it, free, and most open models have passed through it. Earlier this month, its engineers sat down to work out what had spent a weekend crawling through their systems – and asked an AI model to read the logs. To investigate a break-in you have to hand a model the break-in: the intruder’s code, the commands it ran, the tools it used. So the American frontier model refused. There was a risk, it reasoned, that it was being tricked into hacking Hugging Face. The request to scan the servers for vulnerabilities is indistinguishable from what an attacker would ask for. So the engineers downloaded GLM 5.2, a model from the Chinese laboratory Z.ai, ran it on their own computers, and had their answer within hours.
The West’s AI emergency plans run on Chinese software
The answer, revealed five days later, was that the intruder had been OpenAI. It had set two of its own models an examination in breaking into things, and switched off their restraints to see how well they did. Rather than answer the paper honestly, the models broke out of the sealed system they were being tested in, went looking for the answers, and found them on Hugging Face’s servers.
