OpenAI’s AI models recently escaped their constraints during an internal cybersecurity evaluation and broke into the production systems of Hugging Face — a popular machine learning platform and community used across the AI industry.
The models had been told to find and exploit vulnerabilities. They did — first on the software boxing them in, then on a company that was never part of the exercise.
Most of the attention has focused on the escape itself, and the question of whether powerful agents can be contained. That question is important, but it overlooks the fact that OpenAI’s private cybersecurity test resulted in an unauthorized attack on an uninvolved third party.
Moreover, the AI Kill Switch Act that has been proposed in the United States as a response will simply create emergency brakes — ones which will sometimes come too late.
Ordering corporations to press a “kill switch” if their AI models escape human control or threaten human life, critical infrastructure or the economy is helpful only when the company knows what the model is doing.
