Kimi K3 is Moonshot AI's most capable model to date: a 2.8-trillion-parameter model and the world's first open-source model in the 3-trillion-parameter class. It is designed for frontier intelligence work like long-horizon coding, end-to-end knowledge work, and deep reasoning. It is also the first open-weights model competing at the GPT 5.6 Sol and Claude Fable 5 tier, and Together AI is working directly with the Moonshot team to serve it.The largest open-weight model releasedThe Kimi team is deeply committed to scaling, and it shows: in nine of the twelve months from July 2025 to July 2026, Kimi models set the upper bound of open-model scale. At 2.8 trillion parameters, K3 is now the largest open-weight model ever released.What is under the hoodTwo architectural updates form K3's backbone, both designed to help information flow more easily through longer sequences and deeper into the network:Kimi Delta Attention (KDA): a hybrid linear attention mechanism that provides an efficient foundation for scaling attention across very long contexts. This is the first Kimi model to support a 1M context length.Attention Residuals (AttnRes): selectively retrieves representations across model depth rather than accumulating them uniformly.Source: Kimi K3On top of that, Moonshot pushed Mixture-of-Experts sparsity further with the Stable LatentMoE framework, efficiently activating 16 of 896 experts. At this level of sparsity, roughly 2% of experts activated per token, routing and optimization become first-order challenges, so several supporting techniques enable stable training at 2.8T scale:Quantile Balancing: derives expert allocation directly from router-score quantiles, eliminating heuristic updates and a sensitive balancing hyperparameter.Per-Head Muon: extends the Muon optimizer to optimize attention heads independently for more adaptive learning at scale.Sigmoid Tanh Unit (SiTU): improves activation control.Gated MLA: improves attention selectivity.How to use Kimi K3 on Together AIThe API is OpenAI-compatible. The snippets below target Together AI and use the official Together Python SDK.