Most "redact PII before the LLM" tools scan the chat message text and stop there. That was fine when an LLM call was one string in, one string out. It is not fine for agents.
An agentic request carries user data in places that are not the message text:
Tool-call arguments. The model emits send_email({"to": "marie.dupont@acme.com", "name": "Marie Dupont"}). The email and name live inside a JSON string under tool_calls[].function.arguments, not in content. A flat-text scrubber never looks there.
Multimodal parts. content is a list of text and image parts, not a string, so a scrubber that expects a string skips it. (The Responses API input list has the same shape problem.)
So I built a reproducible benchmark to measure how much PII survives in these places, and how the common drop-in approach (a vanilla Presidio pass over the message text, which is what most guardrails ship) compares to a structured-aware one.
