I Built a Security Tool That Proves Its Own Exploits — Then Got a Better Threat Model in the Comments

Automated offense has one embarrassing failure mode: it lies to you about winning.

Point a tool at a target, and the naive success check is a substring match — see uid=0(root) in the response, call it a shell. But a service banner can print that. A tarpit can stream it on connect. And the moment your success signal is wrong, everything downstream inherits the lie: the report, the "which hosts are owned" state, the next move. You get a confident engine that's confidently wrong.

Here's how I made mine prove it instead — what worked in a live run today, and the sharp reader feedback that already made the design better.

The idea: make the target echo a secret it couldn't have guessed

Borrow the oldest trick in authentication. Before each attempt, the orchestrator mints an unpredictable per-attempt nonce and injects it. The delivered command has to send that nonce back: