Alibaba’s Qwen team has released Qwen3.8-Flash-Next, an open-weight multimodal Mixture-of-Experts model designed to preview the architecture for the upcoming Qwen4 series. The checkpoint pairs a 125B backbone with a 51B N-gram embedding table and a 4B multi-token prediction module, activating only 6B parameters per token.

  • Architecture: Uses a hybrid of Gated DeltaNet (linear attention) for three of four layers and Qwen Sparse Attention for the fourth, alongside Gated Residual streams and Muon optimizer.
  • Benchmarks: Achieves 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 91.9 on LiveCodeBench v6, and 95.7 on MathVision with code interpreter.
  • Efficiency: Training cost is roughly one-ninth that of Qwen3.7-Plus; serving shows up to 10.2x prefill throughput increase at high prefix-cache hit rates.
  • Context & Storage: Supports 262,144 tokens natively (extensible to 1M via YaRN); FP8 checkpoint is 172.78 GiB, requiring multi-GPU deployment.

The model serves as an early architectural preview for Qwen4, offering significant cost reductions and speed improvements while maintaining competitive performance on coding and reasoning benchmarks.