Someone on r/LocalLLaMA asked a question this week that turned out to be harder to answer than it looks: which open-source coding harness lets a local model and a cloud model work on the same task, together, in one loop?
The answers he got were a proxy and a shrug. He rejected both, correctly. So here is the actual answer, with the config knobs, and an honest account of the part that does not exist yet.
Why a gateway is not the answer
The first suggestion for this is always LiteLLM, or Bifrost, or whatever OpenAI-compatible proxy is current. Those are good at what they do. They are not what the question is asking for.
A proxy routes on what it can see in the request: the model name you asked for, weights you configured, health checks, budgets. It sits below the agent and has no idea whether the call it is forwarding is a throwaway commit message or the one architectural decision in the run. Point an agent at a gateway and you get failover and cost control. You do not get a local model and a cloud model dividing labour on a task, because nothing in the request says what the task needed.
