Originally published on hexisteme notes.
I run a small ledger-driven pipeline for the consequential decisions in my agent fleet — money, technology adoption, anything load-bearing — and the single rule that has done the most work in it is almost embarrassingly simple: no claim is accepted without an explicit falsification condition. Not "high confidence." Not a percentage. A concrete, observable event or threshold that, if it happens, proves the claim wrong.
I got here by watching my own agents produce confident-sounding numbers that were really just fluent language modeling wearing the costume of analysis. "There's a 70% chance this stock moves up." "This approach will probably work." These read like conclusions. They aren't — they're pattern completion. The model has learned that a confident numeric claim sounds authoritative, and RLHF has rewarded exactly that pattern. The fix isn't better prompting. It's a structural constraint on what counts as an acceptable claim in the first place.
Karl Popper's falsifiability criterion, applied to AI agent outputs: a claim that cannot be falsified isn't a prediction, it's a narrative. For an agent making consequential decisions, every accepted claim has to answer one question — under what observable conditions would this claim be false? Without an answer, the claim can be noted or logged as uncertain, but it cannot be the basis for a decision.
