How Do You Choose Which LLM to Use in Production? I'd Love to Hear Your Process.

The more models I test, the less comfortable I am answering the question: Which LLM should I use?

A model can look like the obvious choice in a benchmark and still be wrong for a real product. Maybe it is too slow. Maybe the price stops making sense at scale. Maybe it follows instructions well in a clean test but falls apart on the messy inputs users actually send.

And even when you make a good choice, it may only stay good for a few months. I have been thinking about this a lot while working on TokenBay, where we connect several models through one OpenAI-compatible API. I spend a lot of time comparing model behavior, pricing, latency, and the friction involved in switching providers. The closer I look, the harder it is to believe in one universally “best” model.

There is only the model that fits your current task and constraints.

So I am curious: How are teams making this decision in practice?

How Do You Choose Which LLM to Use in Production? I'd Love to Hear Your Process.

Related reading

LLM Model Selection Matrix: Pick the Cheapest Reliable Model for Each Feature

Optimizing LLM Model Performance: Best Practices and Techniques

Choosing the Right LLM for Your Agent: A Builder's Comparison Framework

The 5 Things Your LLM Benchmark Misses That Actually Decide the Winner

I Benchmarked 3 Local LLMs on My Laptop — Here's What the Numbers Actually Show

I A/B tested 4 LLMs on the same 500 queries. The results surprised me.