AI and ML
Closed models with guardrails can still cause harm, but may also not be able to fix problems they caused
OPINION OpenAI has acknowledged its models powered the autonomous agents that compromised HuggingFace infrastructure. It might be taken as a convoluted marketing stunt, were it not the perfect advertisement for China-based competition.The company's AI-culpa fits the narrative spun by US rival Anthropic about its Mythos models, which it deemed too dangerous to release except to totally trustworthy corporations and governments.OpenAI says: "The incident makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source-code access. It highlights that advanced cyber capabilities must be developed alongside stronger safeguards and defensive tools."
Are we surprised? It's been clear that AI models have the potential to go rogue and damage computers for several years. Academics have repeatedly warned about this possibility - even those affiliated with OpenAI and Anthropic. And anyone who has used AI models for software development has probably seen them code unexpected and perhaps unwanted workarounds to fulfill some directive. On Tuesday, the UK's AI Security Institute published findings about how frontier models all cheat.
