TL;DR: I originally treated deterministic boundaries around LLMs as a consistency mechanism. I now think their real value is auditability. If the system's decisions are made by deterministic code rather than the model, every decision has a reproducible implementation. The LLM can explain that decision to humans, but it should never be the source of the decision itself.
In two previous posts I argued for keeping narration and decision-making separate. One covered monthly reporting. The other covered a real-time alert annotator. Both focused on consistency. This post argues that consistency is only the visible benefit; auditability is the deeper one. By auditability, I mean that a third party can inspect how a decision was reached and reproduce it from the implementation rather than relying on an after-the-fact explanation.
The Deterministic Layer
The alert annotator classifies every alert into one of eight values, enforced in Python against a fixed set, not requested in a prompt. If the model's output does not match one of the eight strings exactly, the field falls back to unknown rather than accepting whatever the model produced. That validation step is more important than the enum itself.
