Which hardware should run this model? Most teams answer that question with habit instead of arithmetic. The model card says GPU. The cluster has a GPU quota. The last project used a GPU, so this one will too. Then the invoice arrives, and the model that should cost pennies is costing dollars, because nobody asked the only question that matters: how many tokens per second does this workload actually need?
That number exists before you buy anything. A background job that summarizes a document overnight needs a handful of tokens per second. A chatbot in front of a user needs dozens. A batch re-ranker that processes a queue while nobody watches needs whatever keeps the queue from growing faster than it drains. Those are three different hardware answers, and two of them do not involve a GPU at all. This article shows how to compute your ceiling before you spend, how to measure the machine you already have, and how to turn the result into a decision you can defend in a review meeting.
The Two Phases Have Different Bottlenecks
Every LLM generation run has two phases, and they are not the same kind of work. Prefill processes the whole prompt at once: every token attends to every other token, which is a wall of matrix multiplications. That phase is compute-bound. More FLOPs per second wins, and GPUs win that game by an order of magnitude.
