Put the LLM last: I replaced a 7B model with a tiny Go classifier

TL;DR: most production AI tasks are not LLM tasks. To triage my email, I replaced a 7-billion-parameter model with a tiny classifier in Go. The rule fits in one sentence. Rules first, a small model next, the LLM only as a last resort. The result: no GPU, sub-millisecond inference, and a cloud call that became rare. Here is how, with the real numbers.

This article is for developers who put an LLM in production and pay the bill. Not a demo.

Most AI tasks are not LLM tasks

In 2026, the default reflex is to wire a big model into everything. A question comes in, you call the LLM. But many tasks do not need it. Filing an email under "work" or "newsletter" is classification. A problem solved for twenty years, long before LLMs.

To classify is to pick a label from a short, stable list. To generate text is something else. The first job needs a small model. The second earns a big one. The rule I defend fits in one sentence. Put the LLM last.

Put the LLM last: I replaced a 7B model with a tiny Go classifier

Related reading

Optimizing LLM Model Performance: Best Practices and Techniques

I don't trust the LLM to classify my email. So I don't let it.

8GB to 70B: A Real Hardware Guide for Local LLMs

Your AI Bill Isn't a Model Problem. It's an Architecture Problem.

Small language models: Rethinking enterprise AI architecture

How LLMs Work: Transformer, Attention & Next Token Prediction Explained