Alibaba Cloud has open-sourced the Qwen2.5 series of decoder-only dense language models, adding three new sizes (3B, 14B, and 32B) to the existing lineup ranging from 0.5B to 72B parameters. This update expands the pre-training dataset from 7 trillion to 18 trillion tokens and integrates technical breakthroughs from Qwen2.5-Coder and Qwen-math to significantly improve performance in coding, mathematics, and general knowledge.
- The Qwen2.5-32B model outperforms Qwen2-72B, while Qwen2.5-14B surpasses Qwen2-57B-A14B in comprehensive evaluations.
- Knowledge enhancement is evident on MMLU benchmarks, with Qwen2.5-7B/72B scores rising from 70.3 to 74.2 and 84.2 to 86.1 respectively.
- Coding capabilities improved substantially, with Qwen2.5-72B-Instruct achieving 55.5 on LiveCodeBench and 88.2 on MBPP.
- Mathematical ability increased significantly, raising MATH benchmark scores for Qwen2.5-7B/72B-Instruct from 52.9/69.0 to 75.5/83.1.
- Human preference alignment improved, with Arena-Hard scores for Qwen2.5-72B-Instruct jumping from 48.1 to 81.2.
- Context length support extends to 128K tokens, and most models are licensed under Apache 2.0.
The release addresses user demand for cost-effective production models in the 10-30B range and mobile-optimized 3B models, offering highly competitive open-source alternatives that match or exceed larger proprietary systems.