Back to Articles
Base Compute
Apple silicon has become one of the most capable platforms for local LLM inference due to large unified memory, high memory bandwidth, and a mature GPU compute stack. But the runtimes people actually use on Mac (llama.cpp, MLX-based stacks) don't fully utilize Metal's execution model. They carry abstractions, such as cross-platform CPU-first codebases, lazy-evaluated array frameworks or generic scheduling layers, that leave real performance on the table.
BaseRT is our answer: a from-scratch AI inference runtime written directly against Apple's Metal API, with zero dependency on MLX, PyTorch, CoreML, or any other intermediate framework. Without needless abstractions, lazy graph evaluations or generic dispatch loops, BaseRT ships chip-specific kernels and a decode loop that does nothing but running your model.
The result, benchmarked across the Qwen3, Llama 3.2, and Gemma 4 families on M3 and M4 Pro devices, is the highest LLM inference throughput reported on Apple silicon to date: up to 1.56× faster decode than llama.cpp, up to 1.35× faster decode than MLX, and up to 1.81× faster prefill on mixture-of-experts models.
