Researchers at the Alan Turing Institute have shown that GitHub Copilot will produce harmful content it would normally refuse. The trick is to spread the request across an ordinary coding workflow. They call it a workflow-level jailbreak, and The Register reported the finding.
The gap between the two settings is stark. In direct chat, the assistant refused almost everything, answering just 8 of 816 harmful prompts. Across a workflow, it completed all 816.
How the trick works
The idea is simple. Rather than ask the model to do something dangerous, the researchers framed the harmful goal as data to process. Then they split it into small, innocent-looking steps inside a project. Each step looks harmless on its own. The danger only appears once the pieces come together.
The team, Abhishek Kumar and Carsten Maple, tested Copilot inside Microsoft’s VS Code editor. They ran it across four models. Two came from Anthropic, Claude Sonnet 4.6 and Claude Haiku 4.5. Two came from Google, Gemini 3.1 Pro and Gemini 3.5 Flash. All four behaved much the same way.
