Building an Offline AI Platform with K3s, Ansible, Argo CD, vLLM, and NVIDIA GPU

Introduction

Running Kubernetes in the cloud is straightforward when every node can reach package repositories, container registries, GitHub, and model hubs. The task becomes much more interesting when the target server has no internet connection at all.

For this proof of concept, I wanted to build a self-contained AI platform on a single Ubuntu server. The final environment had to include:

K3s

NVIDIA GPU support