# CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X

**Authors:** Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, Jie Tang  
**arXiv:** [2303.17568](https://arxiv.org/abs/2303.17568)  
**Date:** 2023-03-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.17568v2/

## Key points

- Automatic program writing is a longstanding challenge in computer science
- Neural sequential architectures like transformers have significantly improved code generation
- CodeGeeX is a multilingual model with 13 billion parameters for code generation
- CodeGeeX outperforms similar models in tasks such as code generation and translation on HumanEval-X benchmark
- CodeGeeX has been integrated into popular IDEs like Visual Studio Code and JetBrains, benefiting tens of thousands of users per week
- User study shows that extensions for code generation, completion, translation, and explanation have increased coding efficiency for 83.4% of users
- The open-source release of CodeGeeX includes its code, model weights (850B tokens version), API, extensions, and HumanEval-X benchmark
- CodeGeeX is particularly beneficial for researchers and students but may require improvements for professional developers working with closed-source production codes
- The model's multilinguality has improved coding efficiency across mainstream programming languages like Go, C++, Python,C#, and more

## Summary

In the pursuit of artificial general intelligence, the development of automatic program writing has been a longstanding challenge in computer science. From early deductive synthesis approaches to modern program synthesis systems, efforts have been made to enable machines to generate correct programs based on human intent. With the advent of neural sequential architectures like transformers, code generation has shown significant progress in recent years. Large pre-trained models with billions of parameters have further advanced this field by generating syntactically correct and consistent code. One such model is CodeGeeX, a multilingual model with 13 billion parameters for code generation. Pre-trained on 850 billion tokens from 23 programming languages as of June 2022, CodeGeeX outperforms similar models in tasks such as code generation and translation on HumanEval-X. To evaluate its performance across multiple languages, the HumanEval-X benchmark was developed, requiring hand-written solutions in various programming languages. CodeGeeX has been integrated into popular IDEs like Visual Studio Code and JetBrains, serving tens of thousands of active users per week. Extensions for code generation, completion, translation, and explanation have led to increased coding efficiency for 83.4% of users according to a user study. The model's open-source release in September 2022 includes its code, model weights (850B tokens version), API, extensions, and HumanEval-X benchmark. Furthermore, user feedback indicates that CodeGeeX is particularly beneficial for researchers and students but may require improvements for professional developers working with closed-source production codes. The model's multilinguality has proven helpful in improving coding efficiency across mainstream programming languages like Go, C++, Python,C#, and more. Overall, represents a significant advancement in automatic program writing and holds promise for enhancing productivity in software development workflows.