OpenAI has trained an elite hacker, then locked it in a cage. Its whole job is to break OpenAI’s own AI. The company says it is too dangerous to let anyone else near it.
The model is called GPT-Red, and OpenAI detailed it this week. It is an automated red-teamer: software that hunts for ways to hijack or sabotage other AI systems, so the holes can be patched before release. Humans have long done this work by hand. It is OpenAI’s deepest push yet into automating its own AI security, and GPT-Red does it at machine speed.
OpenAI aimed it at prompt injection, where hidden instructions, buried in an email, a web page, or a file, trick a model into doing something it should not. Then it set the hacker loose on real targets.
The training dojo
GPT-Red learns by fighting. OpenAI put it in a self-play loop against a squad of defender models. GPT-Red is rewarded for landing an attack; the defenders for fending one off. As the defenders wise up, GPT-Red must invent nastier tricks. OpenAI says it poured some of its largest ever compute runs into the model, an amount it calls unprecedented for safety work.
