Originally published on hexisteme notes.
I keep my operating judgment in small, versioned files — one file per decision, each with a status, a list of what it supersedes, and a change history. It's a lightweight way to stop re-litigating the same argument every few weeks: write the judgment down once, and the next time the situation comes up, read the file instead of re-deriving the answer from scratch. Most of the time this works exactly as intended — a rule gets written, a rule gets proven wrong, a newer rule replaces it, done. This is the story of the one time that didn't happen, and why "didn't happen" doesn't look like an error. It looks like a rule sitting there, marked active, doing precisely what it was told to do.
The gate
One of my rules governed how to use a panel of eight simulated reviewers I sometimes convene for open-ended judgment calls — different personas critiquing the same plan from different angles. The panel's response includes, among other things, an "internal consistency" score: roughly, how much the eight answers agree with each other. Early on I wrote a gate around that number: if internal consistency comes back under 70%, don't adopt the panel's conclusion on its own. Low agreement, the reasoning went, meant the panel hadn't converged on anything trustworthy.
