Last month I gave an agent full shell access on a side project, stepped away, and came back to find it had run npm install on a package I didn't recognize — something pulled from a typo-squatted namespace with a name close enough to fool it. Nothing bad happened, as far as I could tell. But I lost twenty minutes auditing my own machine instead of shipping anything, which is the opposite of what the tool was supposed to give me.
Claude and GPT-5-class models are genuinely solid at working through multi-step coding tasks now. The problem was that nothing sat between "the model decided to run a command" and that command actually executing, on my laptop, with my permissions.
Everyone wants to talk about which model reasons best. Almost nobody asks what happens the day the best-reasoning model is confidently wrong.
Two failure modes
Agents fail in ways that tend to get lumped together but really aren't the same thing.
