# From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence

**Authors:** Jian Yang, Wei Zhang, Shark Liu, Jiajun Wu, Shawn Guo, Yizhi Li  
**arXiv:** [2511.18538](https://arxiv.org/abs/2511.18538)  
**Date:** 2025-11-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2511.18538v1/

## Key points

- Large language models (LLMs) have revolutionized automated software development by enabling the direct translation of natural language descriptions into functional code.
- Commercial tools like Github Copilot, Cursor, Trae, and Claude Code have facilitated widespread adoption of LLMs in software development.
- Evolution from rule-based systems to Transformer-based architectures has significantly improved performance metrics, with success rates on benchmarks like HumanEval surpassing 95%.
- The paper provides a practical guide through analytic experiments focusing on code LLMs, covering the complete model life cycle from data curation to post-training techniques.
- General LLMs such as GPT-4 and specialized code LLMs like StarCoder and QwenCoder are analyzed in detail.
- The authors offer valuable guidance on leveraging code intelligence in modern software development practices, addressing design decisions, trade-offs, and nuances of code intelligence capabilities.
- Considerations around code correctness, security implications, contextual awareness within large codebases, and integration with development workflows are discussed.
- Promising research directions aligning with practical needs in leveraging code intelligence are highlighted.
- Comprehensive experiments cover various aspects such as code pre-training methodologies, supervised fine-tuning strategies, reinforcement learning approaches, scaling laws, framework selection criteria, hyperparameter sensitivity assessments,model architecture comparisons,and dataset evaluations.

## Summary

In their paper titled "From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence," authors Jian Yang, Wei Zhang, Shark Liu, Jiajun Wu, Shawn Guo, and Yizhi Li delve into the transformative impact of large language models (LLMs) on automated software development. These LLMs have revolutionized the field by allowing for the direct translation of natural language descriptions into functional code. This has led to widespread commercial adoption through tools like Github Copilot (Microsoft), Cursor (Anysphere), Trae (ByteDance), and Claude Code (Anthropic). The evolution from rule-based systems to Transformer-based architectures has significantly improved performance metrics. Success rates on benchmarks like HumanEval have soared from single digits to over 95%. The authors provide a comprehensive synthesis and practical guide through a series of analytic and probing experiments focused on code LLMs. They systematically examine the complete model life cycle from data curation to post-training techniques such as advanced prompting paradigms, code pre-training, supervised fine-tuning, reinforcement learning, and autonomous coding agents. The paper meticulously analyzes both general LLMs like GPT-4, Claude, and LLaMA as well as code-specialized LLMs including StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder. Through their detailed exploration and insights,the authors offer valuable guidance for navigating the complexities of leveraging code intelligence in modern software development practices. They critically examine techniques,d design decisions,and trade-offs associated with these models while shedding light on the nuances of code intelligence capabilities. Furthermore,the authors address the research-practice gap between academic benchmarks and real-world deployment challenges in software-related code tasks. This includes considerations around code correctness,s security implications,c contextual awareness within large codebases,a and seamless integration with development workflows. The authors also map out promising research directions that align with practical needs in this domain. Lastly, a series of experiments are conducted to provide a comprehensive analysis of various aspects such as code pre-training methodologies, supervised fine-tuning strategies, reinforcement learning approaches. This covers topics like scaling laws, framework selection criteria, hyperparameter sensitivity assessments,model architecture comparisons,res,and dataset evaluations.