Reliable AI Agent Control Flow: Keep the State Machine Out of the Prompt
Picture the failure that keeps me up at night. An agent reports that a job failed. The job did not fail. The work went through cleanly, every field extracted, the output sitting right there, correct. And the agent routed itself to the error state and stopped, calm as anything, as if it had done its job. There is no diff to look at. The code did not change. The config did not change. The transition that misfired lives in neither place. It lives in the prompt, as a few lines of English telling the model which state may follow which, and somewhere upstream the model that reads those lines got quietly updated. The machine you deployed is not the machine you are running. Nobody touched it. It drifted out from under you.
I have not had that exact night land on me, and I am writing this so that it never does. But it is not a hypothetical I had to strain to imagine, because the ingredients are sitting in plain sight in agent codebases everywhere. Once you see the shape of it, you stop being able to unsee it, and I landed on a boundary I now treat as non-negotiable: the state machine that governs which step runs next belongs in ordinary code, not in a prompt. State machines are deterministic by definition. LLMs are not. Putting safety-critical control flow inside a model's instruction-following behavior is putting load-bearing logic on a non-load-bearing surface. The fix is mundane. Keep the machine in code and let the model work inside the steps.
