Back to Articles
When a lab releases open weights, there is often a gap between the checkpoint being available and the model running fast on the hardware people own. Runtimes need new kernels, quantized checkpoints need to be verified, and per-device tuning is engineering effort that only the most popular model and hardware combinations receive. For everything else, users have to run models slowly.
The Base Optimization Stack (B:OS) closes that gap automatically. It is a single pipeline that takes a model from open weights to a tuned BaseRT release for a specific device, with the porting and performance tuning done by our AI research agents working under strict constraints. In this post we describe the pipeline and demonstrate it on NVIDIA's Nemotron 3 Nano, a hybrid mixture-of-experts (MoE) model whose architecture BaseRT did not support previously. Starting from scratch, the pipeline produced the fastest inference for Nemotron 3 Nano on Apple silicon: 1.90× llama.cpp and 1.43× MLX on decode, and 1.39-1.76× and 1.98-2.55× respectively on prefill.
Pipeline: Base Optimization Stack
A model enters the pipeline as open weights and leaves as a BaseRT release tuned per model and per device. Every stage is driven by research agents working inside a harness that controls the git history and a global database: every modification is one commit and one benchmark evaluation, so every number is fully reproducible. The pipeline is the following:
