Originally published on hexisteme notes.
I was about to harden a new agent whose whole job is to turn "should I adopt this library, model, or tool" into a deterministic, auditable verdict instead of a vibe — gates, grades, falsifiers, a learning ledger. Before trusting it with that job, I wanted a design review nobody could dodge. My default pattern was "ask my main coding assistant to look it over," which has the same structural problem as a same-family writer reviewing its own writing: builder and checker share the same blind spots. So this time three roles — Questioner, Answerer, and adversarial Verifier — rotated through three reviewer groups in every possible assignment, across six rounds. Three roles into three groups is exactly six permutations, and I used all of them, so no group ever sat as the permanent judge.
The setup: eight targets, six dimensions, three groups
The system under review had eight discrete pieces worth judging, pulled from its own codebase rather than picked after the fact: identity and boundaries (what separates a verdict-making agent from a plain fact-gathering one), four type-level invariants blocking an unverified claim from being laundered into a confirmed fact, five deterministic scoring gates that only score fact-labeled evidence, the grade decision and hard-gate demotion logic built on those gates, automatic derivation of the conditions that would prove a verdict wrong, a provenance parser with a host whitelist for fact-grade sources, a learning ledger checking whether its own confidence is honestly calibrated, and the CLI/bus/config surface a human touches. Each got judged on six dimensions: interesting to judge, useful downstream, complete against its own spec, coherent with its docs and siblings, reliable — reproducible, tested, falsifiable — and actually serving the system's purpose.
