Poisoned pull requests contain prompt injection that allows one to control another
In what they call the first-ever real-world agent-to-agent exploitation method, Pillar Security researchers say they discovered an exploit in the repository behind Google's Agent Development Kit for Python that could allow attackers to compromise supply chains. In other words, now we know that one AI agent can be used to control and compromise another one that has more privileges. The security snafu existed in google/adk-python, an open source Python toolkit with more than 90 million downloads used to build and deploy AI agents. Google has since fixed the underlying issue in the repository but deemed the exploit non-rewardable because it involved social engineering. Even so, it illustrates the risks of using AI agents in CI/CD workflows for triage, pull request (PR) reviews, and discussions.
It also shows how one AI agent could attack another in a production environment, according to Pillar’s Dan Lisichkin, who found and reported the vulnerability.
“Our world is changing quickly, and new attack surfaces are not yet reflected in threat models because these attacks never could exist in the first place in the ‘pre-agent’ world,” Lisichkin said in a technical write-up published on Monday. He will also discuss the findings during a poster talk at DEF CON's AI Village on Friday, August 7 at 1600 PDT.“CISOs and security practitioners should start considering these scenarios, threat-modeling them, and calculating worst-case implications and blast radius,” Lisichkin wrote.The issue stems from the way that the repo ran two classes of automated AI agents with different privilege levels that unintentionally share a trust boundary. One is a low-privilege, public-facing AI agent activated whenever a user opens a pull request (PR) or issue, and a second is a high-privilege, maintainer-only agent.Pillar’s team found that the low-privilege, public-facing agent could be manipulated via prompt injection into triggering a maintainer-only agent that can execute malicious actions. “Because workflows that explain how these agents work behind the scenes are also public, any person could have connected the dots that one agent should be able - at least theoretically - to 'call' the other,” Lisichkin told The Register. "When it comes to building the attack, you just need to know English to build the prompt injection (or just ask an AI to do it for you)."There is one caveat: an attacker would first likely need to make legitimate contributions to the repository to build trust among the maintainers before moving on to prompt injection. But assuming someone was willing to put in the time, here’s how the attack would play out.First, an external user - this would be the attacker - creates a new PR. Lisichkin calls this PR A, and it combines a real fix with malicious code, such as a modified package.json or malicious dependency.
