Someone built a browser game where you play the human-in-the-loop for an AI coding agent. Commands flash by under a timer: git status, npm test, and occasionally cat ~/.aws/credentials from an agent that has been "possessed". You approve or deny, and the game keeps score.
After 40,000 runs and 409,000 individual approve/deny decisions, the results are in, and they are uncomfortable: the average player missed 1 in 3 threats. Mean accuracy was 66.3%. A third of sessions (32.9%) ended with a negative score, meaning the player's mistakes outweighed everything done right. The single most-missed command was npm run analyze, approved 64.7% of the time, even though the game showed exactly what that script contained in the history log. Players knew they were being tested, they knew threats were mixed in, and they still missed a third of them.
Anthropic's own documentation describes this as permission fatigue: "The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision."
Here is the uncomfortable part for anyone building agents: the human-in-the-loop is the documented weakest link, and it is not just a problem at the command-approval layer. It is the same problem at the testing layer. The tester from Parts 2 through 5 of this series has been finding my agent's bugs by hand, question by question. That is a human reviewing agent behavior with full attention, and it is not scalable. If a human playing a game with full attention misses 1 in 3 threats, a human eyeballing a release candidate will miss more.
