My first Ollama setup felt fast. One evening, one model, one question, a clean answer streaming back at a speed that made cloud AI feel unnecessary.

A few days later, with a bigger model and a longer question, the cursor crawled. No error. No warning. Just slow, in a way that made me suspect the model, the network, my own question — everything except the actual cause, which turned out to be almost embarrassingly mundane.

Here's what was actually going on, and the two things that caused it.

The setup

Ollama runs as its own service on Proxmox, with the GPU passed through instead of running on the CPU alone. The whole point of local AI, for me, is that no request ever leaves the house — but without usable speed, that's a nice principle nobody actually uses day to day.