OpenAI has disclosed details of GPT-Red, an internal automated red-teaming model that scales prompt injection vulnerability discovery with an aim to fix issues before the tools are deployed widely.
"GPT‑Red is a strong red-teamer, and our previous models are highly vulnerable to its prompt injection attacks," the artificial intelligence (AI) company said. "We use GPT‑Red to adversarially train GPT‑5.6, making it much more robust to prompt injections."
The model works just like a human red-teamer. It sends a prompt, monitors how a GPT model responds, and iterates its way towards a malicious goal, such as uploading sensitive data to an external server.
The development comes as adversarial prompt injections continue to be a persistent thorn in the flesh of large language models, which can be tricked into executing a carefully crafted instruction that can produce undesirable consequences.
As agentic systems continue to be hooked to third-party data sources through web browsers, connected apps, local files, and other tools, they have also broadened the attack surface and presented more pathways for bad actors to influence the outcome of a model by embedding malicious prompts within seemingly harmless content that's fed as input. This can take the form of an email, a web page, a tool response, or a code repository.
