Deploy Qwen2.5-1.5B-Instruct on a Kubernetes GPU node with vLLM, expose it as an OpenAI-compatible API, and verify it with a real curl request.
Series links
Part 1: Everything You Know About Scaling Web Apps Breaks When You Serve an LLM
Part 2: The Request Is the Wrong Unit of Scale for LLMs on Kubernetes
Part 3: How Do You Fit a Trillion-Parameter Model Into a Kubernetes Cluster?
Part 4: Before the Pod Starts: GPU Node Setup for LLMs on Kubernetes
Introduction: The Day Your Demo Dies Every LLM engineer has a moment like this. Your demo works...
As large language model (LLM) inference workloads grow in complexity, a single monolithic serving process starts to hit its…
A Blog post by Daya Shankar on Hugging Face
Running Chinese LLMs at Scale: A Cloud Architect's Notes I want to talk about something I've been...
Phase 2 of the de-swarm project — how I turned a 3B text-to-SQL model into a production API for...
TL;DR: Kubernetes schedules LLM workloads well, but it does not give them the isolation boundary they...
