I watched a test agent try to delete a PostgreSQL table. It had the credentials. It had the intent. The only thing stopping it was a single line in the system prompt that said "you are a read-only assistant."
That line held. But I've seen enough close calls to know that a prompt is not a safety net.
Every week there's a new horror story. An agent that wiped a production database. An agent that mailed 10,000 customers the wrong message. An agent that ran up a $50,000 API bill in an hour. The common thread is not malicious intent. It's architecture that treats the LLM as a trusted operator instead of a powerful but unreliable intern.
The answer is not "don't use agents." The answer is "use agents with the right guardrails." I've built systems that use LLMs for scoring, generation, and structured extraction at production scale. Here are the patterns that keep agents from doing damage.
Default to Read-Only
