Z.ai released GLM-5.3-Flash, a 320B-parameter multimodal MoE model with 18B active parameters, while Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125B model with 6B active parameters. Despite independent development, both teams adopted nearly identical architectural configurations.
- Both models use a 3:1 hybrid of linear and full attention layers to balance efficiency and precision.
- Context is compressed by a factor of 4 via a learned indexer that caps the sparse attention budget at 2048 tokens.
- The single residual stream is replaced by four parallel gated branches to improve flow control and reduce memory overhead.
- Training utilizes the Muon optimizer with fused projection matrices split into independent transformations before orthogonalization.
- The teams disagree on positional encoding: GLM drops RoPE for NoPE, while Qwen retained RoPE after NoPE variants failed to stop generating post-training.
This convergence suggests a shared industry direction for efficient long-context modeling, though MiniMax remains a dissenter by rejecting linear attention due to reasoning deficits.