Every AI email product shipped in the last two years has the same architecture, and it is the wrong one.
The model reads your mail. The model decides what matters. The model drafts a reply, and in the ambitious ones, the model sends it. Somewhere in the system prompt there is a paragraph asking it politely to check with you first.
That paragraph is the entire safety story. That is the bug.
The bug is not that models hallucinate
Everyone knows models hallucinate. That is priced in. The bug is subtler and worse: these systems give the model authority, then try to constrain it with instructions.
