Quick story. I run a small homelab — one box, an NVIDIA card, around ten Docker containers, and a...
Quick story.
I run a small homelab — one box, an NVIDIA card, around ten Docker containers, and a couple of local model servers (Ollama mostly, vLLM when I'm playing around).
Every "why is this model OOM-ing" turned into the same five minutes of archaeology:
nvidia-smi → pick a PID
ps -o cgroup -p → find the container ID
Running a Tesla P40 for LLM inference. Why I ditched GPU passthrough for host-level drivers to stop the constant Proxmox crashes.
I was tired of running 500MB Docker containers just to chat with a local LLM. So I built a 3.5MB native Linux app for Ollama.…
Most people trying to run Mixtral or DeepSeek-V3 locally hit the same wall: they don't have 80GB of...
Not a GPU unboxing. A real look at what 96GB VRAM enables for multi-model agent pipelines — and where it still hits its limits.
A few weeks ago, I had a classic "works on my machine" moment. I had built a nice RAG prototype...
I've spent the last three months running production workloads across all four major Chinese AI model...
