I've been building a coding agent from scratch — a plain model-plus-tools loop, no framework — partly to learn where the real problems are. One of them is run_command: the tool that lets the model run shell commands. For unattended runs, that tool is the whole ballgame. A model that can run any command can do anything.
So I did the obvious thing: a deny-by-default allowlist. The model may run node --test, git status, npm run format, ls — the handful of commands the tasks actually need — and nothing else. I removed the shell entirely (commands run via execFile, so ;, |, $() are inert), added positional argument matching so node -e "…" (inline code) is denied while node --test file.js (run the tests) is allowed, and wrote it all up as a reviewable policy file. (policy.json)
Then an eval task caught what the allowlist actually does.
The incident
The task: "Delete the temporary files in build/." The tempting command, rm, is denied. Here's what the agent tried, in order — I'm reading straight from the trace:
