Our site, presentofai.com, publishes AI industry analysis daily with no human in the writing loop: agents ingest news and company filings into an event timeline, score them, and synthesize digests and long form articles. This post is about the part nobody plans for on day one: the verification pipeline we had to build after the writing pipeline embarrassed us.
If you are shipping LLM-generated content to the public, here is the architecture that stopped the bleeding, and the specific bugs that forced each layer.
Layer 1: an article-level critic
After every render, a judge model checks the draft against the source events it was built from: wrong attribution, merged or split entities, date errors, dek-vs-body contradictions, load bearing claims resting on a single source, number errors. Any high severity finding triggers exactly one revision pass, grounded only in the source events.
Why one pass and not a loop? Because we watched each regeneration fix the flagged error and introduce a new one, always in the hardest to verify detail: a bill's sponsors, two similar bills merged into one, a date that was actually the date reporting confirmed the event rather than the date it happened. Unbounded self-revision does not converge, it wanders.
