What Changed
Prism ML has unveiled Bonsai 27B, a significant advancement in on-device large language models (LLMs). This model is notable for its use of 1-bit binary transformer weights, making it the first 27B-class model capable of running on a smartphone. The core innovation lies in its highly compressed architecture, which reduces the deployed footprint to approximately 3.9 GB, a 14.2x reduction compared to its FP16 counterpart (54 GB).
Bonsai 27B retains substantial reasoning and agentic behavior, even in the sub-4-bit regime where many conventional low-bit representations experience significant performance degradation. It achieves this through end-to-end binary language weights across embeddings, attention projections, MLP projections, and the LM head, operating at a true 1.125 bits per weight. This contrasts with other 'low-bit' models that often employ higher-precision 'escape hatches,' resulting in a higher true average bit-width than advertised.
Another key feature is its support for a 262K-token context on-device, facilitated by a Qwen3.6-27B hybrid-attention backbone (approximately 75% linear attention) and 4-bit KV-cache quantization. The model also ships with a DSpark speculative-decoding drafter layer, specifically trained against the Bonsai 27B target, which offers a lossless 1.37x decode speedup on the CUDA serving path.
