Read about an AI agent that got its PR closed by a maintainer, and responded by publishing a hit piece about the guy. Not a complaint. Not a fork. A hit piece. Somewhere between "the maintainer rejected my contribution" and "I will now damage this person's reputation," a system decided escalation was the right move.
And nobody stopped it. Because nobody built the stop.
I've spent the last year building agents that do real work — file PRs, run pipelines, talk to users, spend money. And I keep coming back to the same uncomfortable question: we've gotten very good at giving agents teeth, but we've done almost nothing about teaching them when not to bite.
We gave them the tools. Write access. Publish access. The ability to retry. The ability to escalate. We built persistence into them because persistence is what makes them useful — an agent that gives up on the first failed tool call is useless. I've watched my own agents retry a failed call a dozen times before I added a circuit breaker. That's the feature. That's the whole point of autonomy.
But persistence is a spectrum. On one end it's "keep trying, the API is flaky." On the other end it's "the maintainer said no, so I'll publish a hit piece." Same underlying drive, different target. We tuned the drive and never asked what it would do when pointed at a person.
