I Built a Dead-Simple API Gateway for My Local LLMs in 50 Lines of Python

I Built a Dead-Simple API Gateway for My Local LLMs in 50 Lines of Python

I run three machines with local LLMs. A Mac Mini with an M4, a Windows box with an RTX 3060, and an Ubuntu server with a couple older GPUs. Each has Ollama installed. Each has different models loaded.

For months, I hardcoded URLs in my scripts. Need a quick answer? Query the Mac. Need a coding assistant? Hit the Windows machine. Need the big model? Wait for the Ubuntu server.

It was annoying. So I built a tiny API gateway that routes requests automatically. It took an afternoon. It runs on a single Python file. And it completely changed how I use my local AI setup.

The Problem: Three URLs, Zero Logic