CPU rightsizing is a solved, well-documented practice. GPU rightsizing is where the real money is now, and almost nobody writes about it, because GPU instances are expensive enough that people are scared to touch them and unsure how. Given how much a GPU box costs per hour, an over-provisioned one is the single most expensive rightsizing mistake in your account. Here is how to rightsize AWS GPU instances without breaking the workload, including the compatibility check that quietly bites people.
Know what each GPU family is for
Rightsizing starts with using the right family, not just the right size. On AWS:
G5 / G6 (NVIDIA A10G / L4): inference, graphics, smaller training. The workhorses for serving models and lighter ML. Cheaper per hour.
P4 / P5 (A100 / H100): large-scale training and heavy inference. The expensive tier, built for jobs that genuinely need the horsepower and interconnect.
