Read this first: You don't need a frontier model to redact PII
Asked to redact a PII-dense document, llama3.1:8b sometimes answers: "I cannot provide information that could be used to identify an individual." That is a 200 response containing a well-formed English sentence. Write it into your destination field and the original document, every name intact, carries on downstream. The record looks processed.
That was the pattern behind most of the problems we hit while benchmarking six redaction approaches. A parser that fails throws. A schema validator that fails rejects. A redaction pass that fails returns a plausible-looking string in the expected shape, and nothing objects.
The fixes below are less about detection quality than about manufacturing the error signal the system does not produce on its own.
The ordering mistake
