AI agents are a confused deputy with the keys to your kingdom

Earlier in June, attackers took control of more than twenty thousand Instagram accounts, including the dormant Obama-era White House account, without writing an exploit or guessing a single password. They opened a chat with Meta's AI support assistant, asked it to attach an email address they controlled to an account they did not own, and requested a password reset to that address. Meta later confirmed what the logs already showed: the assistant behaved exactly as designed, while a separate part of the system was supposed to verify that the email belonged to the account, and that check never ran.Calling this an AI mistake misses what happened. The assistant carried out a valid sequence of permitted operations for whoever was talking to it. What would have stopped the attack was a person: a support worker who saw a stranger rerouting a celebrity's recovery email, sensed something was wrong, and refused.A large share of real-world authorization was never written as software at all. Instead, it lived in the discretion of whoever stood between a request and the system, and everything behind them was built assuming that discretion would always be there. Put an agent in that seat and discretion vanishes, while nothing and nobody downstream notices. The agent does not bypass your security model, it just exposes the part of it that was a person.Security has a precise term for what Meta hit: the confused deputy. A process holding real privileges is talked by a less-privileged party into using those privileges on its behalf. It's the night guard who unlocks the vault for anyone who calls and says the boss sent them: he's got the keys, they've got a good story. The canonical 1988 case was a compiler that could write to a protected billing file. A user who could not write there asked the compiler to do it for them, and it complied, because it had the authority and never asked whose request it was serving.An LLM agent is one of these by construction. Its interface is natural language, which carries no notion of who is authorized to do what, and the model's whole job is to turn a plausible-sounding sentence into a tool call. A direct API request at least brings the caller's identity along with it. A sentence does not, so unless that identity is reattached before the call fires, the agent acts on its own authority and the requester's permissions never enter the picture.Agents also cannot reliably separate instructions from data. Everything in the context window reads as potential instruction: the user's message, a retrieved document, the body of an email the agent was asked to summarize. A support bot that resets a password because a convincing chat told it to will just as readily follow a command hidden inside a file it was handed to process. The VPN trick that defeated Meta's geolocation check is the crude version of this. The sophisticated versions, where the malicious instruction is smuggled in through content the agent ingests, are already being documented as the dominant class of agent attack.The Instagram bot could reset passwords, and that's a serious breach, but a bounded one. The agents shipping now are not bounded that way. In the same week Meta disabled the support tool, it launched its Business Agent, which books appointments, qualifies leads, closes sales, takes payments, and connects to systems like Shopify and Zendesk to act on a company's behalf. Run the same confused-deputy logic through a payment API and a CRM and the failure is no longer a stolen account. It is a refund sent to the wrong party, an order rerouted, a price overridden, a customer record edited, each one a legitimate operation the agent was authorized to perform for whoever asked.The market is outrunning the security model. Gartner has projected that 40% of enterprise applications will include task-specific AI agents by the end of 2026, up from under 5% at the start of it. Most of those deployments will inherit the same assumption Meta's did, which is that whatever sits at the far end of a privileged action has judgment.A more capable model behind the same workflow would have handed over the same accounts with better grammar, which is exactly why the model cannot be the place authorization lives, being the part an attacker controls. The decision to allow an action has to be made outside it, by a policy layer that checks who is actually behind the session before anything runs. Meta's assistant never established that the person it was talking to owned the account before it rebound their recovery email.In a couple lines of code, what shipped looked something like this. The agent could call the function, and being able to call it was the whole authorization:def add_recovery_email(account, new_email):

Raccontata daforbes.comstackoverflow.blog