# Qwen2.5-Coder Technical Report

**Authors:** Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, An Yang, Rui Men, Fei Huang, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, Junyang Lin  
**arXiv:** [2409.12186](https://arxiv.org/abs/2409.12186)  
**Date:** 2024-09-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2409.12186v1/

## Key points

- The Qwen2.5-Coder series is a significant advancement in code intelligence technology
- Two distinct models: Qwen2.5-Coder-1.5B and Qwen2.5-Coder-7B, built upon the Qwen2.5 architecture
- Pretrained on an extensive corpus of over 5.5 trillion tokens for remarkable code generation capabilities
- Meticulous data cleaning processes, scalable synthetic data generation techniques, and balanced data mixing strategies were employed during development
- Outperformed larger models of comparable size in more than 10 benchmarks across various code-related tasks
- Poised to redefine research in code intelligence and set new standards for excellence in the field
- Permissive licensing approach expected to catalyze broader adoption among developers for real-world applications

## Summary

The Qwen2.5-Coder series, introduced in the technical report authored by Binyuan Hui, Jian Yang, Zeyu Cui, and other esteemed researchers, represents a significant advancement in code intelligence technology. This series builds upon its predecessor, CodeQwen1.5, with two distinct models: Qwen2.5-Coder-1.5B and Qwen2.5-Coder-7B. Leveraging the robust foundation of the Qwen2.5 architecture and pretrained on an extensive corpus of over 5.5 trillion tokens, these models showcase remarkable code generation capabilities while maintaining a high degree of versatility. The development of the Qwen2.5-Coder series involved meticulous data cleaning processes, scalable synthetic data generation techniques, and balanced data mixing strategies to enhance performance across various code-related tasks. Through rigorous evaluation on more than 10 benchmarks encompassing code generation, completion, reasoning, and repair tasks, the Qwen2.5-Coder models consistently outperformed larger models of comparable size to achieve state-of-the-art (SOTA) results. The release of the Qwen2.5-Coder series is poised to redefine research in code intelligence by pushing boundaries and setting new standards for excellence in the field. Furthermore, with its permissive licensing approach, these advanced models are expected to catalyze broader adoption among developers for real-world applications. In conclusion, the collaborative efforts of the talented team behind the Qwen2.5-Coder Technical Report have culminated in a groundbreaking advancement that not only showcases cutting-edge capabilities in code generation but also paves the way for widespread implementation in practical scenarios across various industries and domains.