The Qwen team has released the Qwen2 series, evolving from Qwen1.5 with five new model sizes: Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, and Qwen2-72B. These models are now available on Hugging Face and ModelScope.

  • All model sizes utilize Group Query Attention (GQA) for faster inference and reduced memory usage.
  • Training data includes 27 additional languages beyond English and Chinese to improve multilingual capabilities.
  • Qwen2-7B-Instruct and Qwen2-72B-Instruct support extended context lengths up to 128K tokens using YARN.
  • The Qwen2-72B model demonstrates superior performance compared to Llama-3-70B and its predecessor Qwen1.5-110B despite having fewer parameters.
  • Significant improvements were made in coding, mathematics, and safety alignment through automated post-training strategies.

The release aims to provide open-source models with state-of-the-art performance across natural language understanding, reasoning, and multilingual tasks while maintaining strong safety standards.