Six months ago I got frustrated with cloud-locked LLM deployment tools. So I built kube-llmops — a complete open-source LLMOps platform for any K8s cluster.
Six months ago, my team needed to deploy DeepSeek-R1 for internal use. We have a Kubernetes cluster — like everyone does in 2026 — so I started looking for tools.
The problem
There are basically three options for running LLMs on Kubernetes:
KAITO (Microsoft) — CNCF Sandbox, 1600+ stars, but Azure-only. We are on AWS.
KServe — CNCF Incubating, solid project, but requires Knative + ISTIO + 5+ other components.
TL;DR: Kubernetes schedules LLM workloads well, but it does not give them the isolation boundary they...
I abandoned this Kubernetes platform on April 4th. 48 days later I rebuilt it: CrashLoopBackOff everywhere → self-service GitOps,…
Repo:github.com/hearth-project/hearth · Apache-2.0 · v0.1.0, alpha. I've been building Hearth, a...
Using Kubernetes is a love-hate relationship. Love, because before it, deploying to production was...
A practical guide to open-source tools for deploying, serving, and running foundation models, from local LLMs to high-throughput…
aws.amazon.comCategory: AWS Deep Learning AMIs
aws.amazon.com