Alibaba’s Qwen team has released Qwen3.8-Flash-Next, an open-weight preview of the architecture it intends to use for Qwen4, carrying 125B parameters but activating only 6B for each token. Its licence may not qualify for the EU AI Act’s open-source exemption.
Alibaba’s Qwen team has published the architecture it plans to build Qwen4 on. Qwen3.8-Flash-Next carries 125B parameters and fires only 6B of them for each token it produces.
The claim is about cost rather than capability. The team says its concern is what architectural choices do to inference bills as agentic jobs with very long contexts become the normal workload.
The comparison it draws is with its own last model. Qwen3.7-Plus holds 397B parameters and activates 17B, so this one runs on roughly a third of the active compute.
Three of the four changes are conventional enough. A new sparse attention scheme works on micro-blocks instead of picking individual tokens, a gated residual mechanism controls what passes between layers, and the training recipe drops batch-size warmup entirely.
