Originally published on tamiz.pro.
You've fine-tuned your model. It runs on your GPU server. Prompt injections are a lab curiosity. But the moment you expose it to real users — with tools, memory, and agent loops — the attack surface explodes. Local doesn't mean secure. In this deep dive, I'll walk you through the architecture, guardrails, and operational practices that turn a fragile lab prototype into a production-hardened self-hosted LLM system protected by GitOps.
Why "Local" Doesn't Mean Secure
Self-hosted LLMs offer privacy, cost control, and compliance advantages. But they also inherit every risk of any production system — plus new ones unique to generative AI. A local model serving over HTTP is just an API server, and API servers get probed, exploited, and exfiltrated. AI agents compound this with tool access, persistent memory, and autonomous action loops.
The lab mindset treats these as features. Production treats them as threat vectors. Bridging that gap requires infrastructure that is auditable, version-controlled, and relentlessly automated — exactly what GitOps delivers.
