Ascend Tribe has released openPangu-2.0-Flash, a Mixture of Experts (MoE) large language model trained on Ascend hardware. The model features 92 billion total parameters with 6 billion activated per token and supports a context length of 512k tokens.
- Total pretraining data consists of 34 trillion tokens.
- Post-training includes unified SFT for slow and fast thinking capabilities.
- Training utilizes multiple specialist RL training and on-policy distillation combining multiple RL specialists.
The model is currently available via GitCode, though it has not yet been uploaded to Huggingface.