The GPU hourly rate is the number everyone compares. It is also the number that tells you the least about what a training run actually costs.
The sticker price, say $2 to $3.50 an hour for an H100 on a specialized cloud, is the visible tip. The real bill is built from three things almost nobody puts on the comparison spreadsheet: the GPU sitting idle, the data you have to move, and the cost of ever leaving. This post breaks down each one, with 2026 numbers, and what you can actually do about it.
1. Idle time: paying full price for nothing
The most expensive line item in most setups is not compute. It is compute you pay for but never use.
The 5 percent problem
