AI agents comprise models, instructions, data, and tools, so thoroughly investigating potential security risks requires evidence from several components. As Datadog teams build AI agents for internal workflows, we use Datadog AI Guard to monitor how they handle each component during a session. We’ve found that application logs may capture an agent’s final API call without showing which prompt, retrieved content, or tool result led to the action. For adequate security monitoring, we needed data that connected the final action with the entirety of an agent’s session.
To understand what led to an agent’s final action, we expanded monitoring to include an inventory of its components and telemetry data across the full execution path. This approach enabled us to track access to sensitive data, record human and agent identities separately, and look for atypical sequences of behavior within a session.
In this post, we’ll explain what we observed at each layer and which telemetry data was most useful for monitoring agent activity. We’ll also look at Datadog Security Labs research and findings from our State of AI Engineering report to provide additional context on related risks and AI engineering trends.
