No Low-Severity Alert Goes Uninvestigated
by Leanne Shapton, Connor Hanify and Sam Pezzino
Databricks ingests petabytes of security logs from a variety of sources including endpoint security tools, cloud activity logs, and threat intelligence feeds into our security lakehouse. Our detection architecture continuously monitors this data for malicious activity. Every identified signal lands in a centralized alerts table, where it awaits review by an Incident Response (IR) analyst.
Finding a real threat in thousands of daily security alerts is the classic needle-in-a-haystack problem. Most teams handle the load by prioritizing alerts by severity. Teams will triage the HIGHs and MEDIUMs, and work through LOWs as bandwidth becomes available.
At Databricks, our IR team handles security alerts across all three priority levels. Historically the team prioritized HIGH and MEDIUM alerts due to the volume and low fidelity of LOW alerts. Every security team at scale faces this same tradeoff: increase analyst headcount, raise alert thresholds and accept blind spots, or find a way to automate the process. We chose the latter, leveraging agents that can reason and execute judgment at scale.
