The Tencent Hy Team has open-sourced Hy4-preview, a new-generation Mixture-of-Experts flagship model comprising 770B total parameters with 49B activated per token. The architecture features 78 layers where the first uses a dense FFN and the remaining 77 use MoE with 256 routed experts, alongside a native MTP layer for speculative decoding.
- Hy4-preview employs Gated DeepSeek Sparse Attention and identity Hyper-Connections to enhance information flow.
- Training data was curated with internal Tencent experts in software engineering, finance, and game development.
- In blind side-by-side evaluations on 203 engineering tasks, Hy4-preview scored 2.99, slightly ahead of GLM 5.3 (2.92) and Kimi K3 (2.94).
- Weights are available in standard and FP8 formats under the Apache License 2.0.
The model aims to improve productivity across complex tasks like long-horizon coding, data analysis, and scientific research, with deployment support for vLLM and SGLang.