Qwen3.8-Flash-Next

Open-weight preview of the Qwen4 architecture, playing the role Qwen3-Next played for Qwen3.5: a multimodal (image/video-in, text-out) MoE with a 125B-parameter backbone, 6B active per token, plus a 51B N-gram embedding table and a 4B multi-token-prediction head (~180B in the safetensors). The 48-layer stack keeps the 3:1 Gated DeltaNet hybrid but swaps gated attention for Qwen Sparse Attention (QSA) — a lightweight per-layer indexer that scores context at micro-block granularity (2,048-token budget), introduced during 256K continued pretraining. Three further changes: a Gated Residual that widens the residual stream into four branches with element-wise read gates and scalar write gates (a simplified Hyper-Connections + GatedNorm); an N-gram embedding layer (20M-entry bigram/trigram table at layer 2, inspired by Gemma 3n per-layer embeddings and DeepSeek Engram) that adds capacity at ~zero per-token FLOPs and can be prefetched from host memory; and a Muon/AdamW split (Muon for attention, GDN and expert matrices; AdamW for embeddings, router and low-rank gates) with a refitted scaling law that removes batch-size warmup. 512 experts, 10 routed + 1 shared active, 262K native context extensible to 1M with YaRN.

Qwen reports roughly 1/3 the training tokens and 1/9 the training FLOPs of Qwen3.7-Plus (397B-A17B), with the base model ahead of it on 8 of 14 pretraining benchmarks; absolute token counts are not disclosed. At 1M tokens the QSA kernel gives 7.6×/4.9× prefill/decode speedups and 8.6× Qwen3.7-Plus's prefill throughput at 90% prefix-cache hit. Post-trained scores: SWE-bench Pro 62.5, SWE-bench Multilingual 81.0, DeepSWE 1.1 58.7, Toolathlon Verified 73.5, GPQA Diamond 91.7, HLE 35.9, LiveCodeBench v6 91.9, AndroidWorld 84.5 — beating Qwen3.8-27B and Qwen3.7-Plus on most rows. AA Intelligence Index v4.3: 40, level with the 2.4T Qwen3.8-Max (40). Thinking on by default with xhigh/medium/low reasoning_effort and preserve_thinking. Weights under the new Qwen Community License 1.0 (MIT-style, but products with >100M MAU or >$20M monthly revenue must display the model name, and MaaS / AI-work-assistant businesses need a separate commercial license). Served on QwenCloud as Qwen3.8-Flash with 1M context by default at $0.15/$0.47 per M tokens; tech report "On the Design of Qwen3.8-Next Architecture" ships as a GitHub PDF (no arXiv ID at release).