Same hardware. Same benchmark. Opposite winner depending on model size. Cross-platform inference numbers on Mac M2 Pro Metal, Linux RTX 2080 Ti CUDA, and Windows RX 6600 XT Vulkan, plus the hardware tier ceiling none of them clear.
AMD RX 6600 XT beat Mac M2 Pro on Llama 8B (+80%) but lost on Phi-3 mini; hardware choice depends on model size versus cache. Local inference hardware selection is model-dependent: Mac wins on cache-fit small models, discrete GPUs on 7-8B price-per-token, nothing viable for 70B+.
I wrote a post yesterday about why GPUs barely help small text embeddings at batch=1. Different workload, same machines. This time I ran a local LLM inference benchmark across the same three boxes. The result complicated my hardware mental model in a way I think is worth sharing.
The setup
Three machines.
A Mac M2 Pro with 16 GB of unified memory, running Metal through llama-cpp-python.
A Linux desktop with an Intel 13700K, 62 GB of RAM, and an RTX 2080 Ti with 11 GB of VRAM. CUDA 13.
Most people were told the RX 580 was dead for AI in 2026. CUDA-only ecosystems, ROCm dropping Polaris...
Most people trying to run Mixtral or DeepSeek-V3 locally hit the same wall: they don't have 80GB of...
A Blog post by Mishig Davaadorj on Hugging Face
I built a peer-memory endpoint so my Mac can query my Linux box's memory store over a WireGuard mesh. The first measurement felt…
No matter what computer you have, there's a model that will run on it. GPU comparison table, budget builds from $0-$2500,…
MI350P packs 144 GB of HBM3e and up to 4.6 petaFLOPS of FP4 grunt into a dual slot card
