Prompt Versioning: How I Learned the Hard Way
Three weeks before a client demo, I pushed what I thought was a minor tweak to a production prompt — changed "respond concisely" to "respond briefly and directly" — and watched our accuracy metric drop from 91% to 67% overnight. No git commit. No record of what it said before. No way to roll back. I spent two days reconstructing the original from memory and Slack messages, and I still don't know if I got it exactly right. That moment is what turned prompt versioning from something I knew I should do into something I actually do.
The Lie We Tell Ourselves About Prompts
There's a mental model most developers carry when they start working with LLMs: prompts are config, not code. You throw them in an .env file or hardcode them in a constants file, tweak them until the output looks right, and move on. The model does the heavy lifting; the prompt is just instructions.
This is wrong, and it costs you eventually.
