The Qwen2 Technical Report introduces the Qwen2 series, a new suite of foundational and instruction-tuned large language and multimodal models ranging from 0.5 to 72 billion parameters. This release includes both dense architectures and a Mixture-of-Experts model, with the flagship Qwen2-72B demonstrating competitive performance against proprietary models across various benchmarks.

  • The base Qwen2-72B model achieves scores of 84.2 on MMLU, 37.9 on GPQA, 64.6 on HumanEval, 89.5 on GSM8K, and 82.4 on BBH.
  • The instruction-tuned Qwen2-72B-Instruct variant attains 9.1 on MT-Bench, 48.1 on Arena-Hard, and 35.7 on LiveCodeBench.
  • The models support approximately 30 languages, including English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, and Vietnamese.
  • Model weights are openly available on Hugging Face and ModelScope, with supplementary code and deployment resources provided on GitHub.

The authors release these weights to foster community innovation and accessibility, providing resources for quantization, fine-tuning, and deployment to facilitate a wide range of applications and research endeavors.