AntLing has released six base model checkpoints for its Ling-3.0-tiny and Ling-3.0-flash architectures, covering pre-trained, mid-trained, and WSM-merged stages. These models have not undergone post-training, providing researchers with flexible starting points for continued pre-training, fine-tuning, and further research.

  • The release utilizes Weighted Checkpoint Merging (WSM) to replace learning rate decay, facilitating offline exploration of different strategies and better suitability for continual pre-training.
  • Ling-3.0-tiny-base features 7.9B total parameters with 1.3B active, delivering comparable or superior performance on most benchmarks compared to Ling-2.5-mini-base, particularly in coding.
  • Ling-3.0-flash-base contains 124B total parameters with 5.1B active, achieving strong results across coding, reasoning, and long context tasks despite being smaller than competitors.
  • A shared training recipe allows the community to validate strategies on the tiny-base model before scaling them to the flash-base architecture.

The release enables domain adaptation in specialized applications such as finance and healthcare, while supporting model behavior studies and MoE research through its sparse activation structure.