Your LLM-based vulnerability scanner just cleared a PR with a real, exploitable bug in it. Not because the scanner is dumb. Because someone wrote a comment specifically designed to talk it out of flagging the code.
That's the finding from researchers behind ALIBI (arxiv.org/abs/2607.24964), an automated attack framework that inserts adversarial natural-language comments into source code to manipulate LLM vulnerability detectors. No behavior change to the program itself. Just text, aimed at the model reading the code, not the compiler running it. Success rate: over 90% against four representative detectors, including frontier multi-agent systems that are supposedly more robust because they reason in multiple passes.
Let that sit for a second. Multi-agent architectures are usually pitched as a defense against exactly this kind of manipulation, more checks, more cross-validation, harder to fool. ALIBI beat them anyway, over 9 times out of 10.
How the attack actually works
The paper describes two mechanisms, both operating purely at the comment/text layer:
