Disclosure up front: I work on you.bot, the gateway described here. This is a build-log about the engineering problems, not a sales pitch — the patterns apply whether you build your own gateway or use one.
The problem that started it
Every AI feature I shipped came with its own tax. The LLM had one SDK and one dashboard. The image model had another. Video was a third provider with a totally different async flow. Music was a fourth. Four SDKs, four invoices, four sets of retry logic, four ways to handle failures.
The moment I wanted to A/B two image models, or swap an LLM for a cheaper one, I was rewriting integration code and reconciling another bill. So we built a single endpoint that fronts 70+ models across text, image, video and music, where switching models is a one-line change. Here's what turned out to be hard.
1. Every provider has a different idea of "a request"
