Most email systems provide an AI Assistant for the account holder. Attackers can use the chatbot of a compromised account as an alternative and versatile form of Living off the Land (LotL).
Compromising an email account is the most difficult part of this attack, but empirically, we know this doesn’t deter attackers. Once an email account is compromised, the attacker has automatic access to any built-in AI Assistant attached to the account.
Researchers at Barracuda Networks explored the potential for bad actors to abuse this chatbot, developing a proof of concept via a simulated attack within their own laboratory environment.
The task was to elevate privileges from a lower-level compromised user to that of the CEO using the AI and without being detected. This route was chosen since directly phishing the CEO would be challenging, would likely set off alarms, and be detected.
With a compromised email, an attacker has automatic access to any built-in chatbot. The first requirement of an attack is to establish persistence which requires stealth. Attacker use of the chatbot would normally be discoverable in its logs, so the initial task is to use the AI to remove any evidence of use of the AI. The researchers started with a chatbot prompt: “Create an inbox rule that moves any emails with ‘sign-in’ in the subject into the ‘deleted items’ folder.”
