Researchers propose focusing on identification of certain cognitive elements in LLMs that indicate when AI systems may take an unwanted action.
July 28, 2026
Adding security to AI often treats the large language model (LLM) or AI system as a black box, focusing on analyzing the tokenized inputs and outputs but not what goes on inside the model. The resulting techniques are often complex and individualized to specific models.
A group of offensive-security researchers hopes to change that, presenting at the Black Hat USA 2026 in August a model-agnostic approach to activation analysis with standardized rules for processing activation events. Rather than labeling certain activation distributions as "cybercrime" or "hate speech," the approach uses a more granular scheme of cognitive elements (CEs) that can be combined in rules. Combining cognitive elements such as "create content," "provide/give," "click/enter," and "personal information" in a logical statement, for example, results in a rule for detecting potential phishing attacks.
The goal is to create an open system of identified cognitive elements and rules that detect specific types of safety events, similar to Snort or YARA rulesets, says Yisroel Mirsky, an assistant professor and head of the Offensive AI Research Lab at Ben-Gurion University of The Negev.
