When it comes to protecting our internal platforms, the threats we defend against on Figma’s security engineering team are shifting constantly. Cloud infrastructure changes frequently, developers adopt new tools, and what people install and run on their laptops is different every quarter (like when your Designer Advocates team starts vibe coding their own apps and automations).Our SIEM, Panther, helps us keep ahead of all these threats by running a wide range of checks across our cloud infrastructure, endpoints, SaaS apps, and identity systems. When it detects a potential issue, it posts alerts into Slack and creates an Asana ticket for on-call engineers to pick up. Historically, being on call involved a massive amount of manual work. Gathering context was the main challenge. We’d have to figure out if the alert resembled something we’d just seen last week, if there was a pending PR that might address the issue, if Slack threads mentioned something related, and so on.Like many teams, we saw LLM model capabilities rapidly accelerate over the past year. We recently shared how Figma stays ahead of vulnerabilities with agents in our codebase, which caught issues in configuration changes to sensitive tools like Okta and AWS thanks to previous investments in storing their configuration as code. But to meaningfully reduce toil and improve how we protected Figma’s internal systems, we needed to go beyond the codebase and build a new system that could help us handle the wide scope of issues our SIEM finds.We started with a narrow goal of building a retrieval layer that could surface prior on-call engineer reasoning when a new alert fired. That project eventually grew into a full agentic system that investigates alerts, queries audit logs, writes code changes, opens PRs, and gets better over time through its own memory. It’s completely changed how our security team works.The RAG layer: Giving alerts a memoryThe first thing we built was a retrieval-augmented classification system on top of AWS Bedrock Knowledge Bases and Amazon Kendra. Again, our initial goal was just to surface historical context about what had happened the last time the same alert fired, and maybe suppress duplicate alerts.When a Panther alert fires, our Lambda handler converts it into a standardized document and indexes it into Kendra. We pull structured fields out of the raw alert payload: IPs from p_any_ip_addresses, actors from p_any_usernames and various provider-specific user fields, AWS account IDs from ARNs. These become searchable Kendra document attributes alongside the alert title, severity, tags, and timestamps:TypeScriptconst attrs: DocumentAttribute[] = [