# Qwen2.5 Technical Report

**Authors:**  Qwen,  :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin  
**arXiv:** [2412.15115](https://arxiv.org/abs/2412.15115)  
**Date:** 2024-12-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2412.15115v1/

## Key points

- Introduction of Qwen2.5 series of large language models (LLMs) catering to diverse needs
- Significant improvement in both pre-training and post-training stages
- Pre-training phase scaled up from 7 trillion tokens to 18 trillion tokens for common sense, expert knowledge, and reasoning capabilities
- Enhanced post-training techniques with supervised fine-tuning using over 1 million samples and multistage reinforcement learning
- Offered in rich sizes including base and instruction-tuned models with quantized versions available
- Demonstrated top-tier performance across various benchmarks evaluating language understanding, reasoning, mathematics, coding, human preference alignment

## Summary

The Qwen2.5 Technical Report introduces the Qwen2.5 series of large language models (LLMs) designed to cater to diverse needs. This latest iteration, Qwen 2.5, represents a significant improvement in both pre-training and post-training stages. The pre-training phase has seen a scaling up of high-quality datasets from 7 trillion tokens to 18 trillion tokens, providing a robust foundation for common sense, expert knowledge, and reasoning capabilities. Post-training techniques have been enhanced with intricate supervised fine-tuning using over 1 million samples and multistage reinforcement learning. To address varied use cases effectively, the Qwen2.5 LLM series is offered in rich sizes including base and instruction-tuned models with quantized versions available. Qwen2.5 has demonstrated top-tier performance across various benchmarks evaluating language understanding, reasoning, mathematics, coding, human preference alignment, among others.