# From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging

**Authors:** Yuling Shi, Songsong Wang, Chengcheng Wan, Xiaodong Gu  
**arXiv:** [2410.01215](https://arxiv.org/abs/2410.01215)  
**Date:** 2024-10-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.01215v1/

## Key points

- Large language models (LLMs) have revolutionized AI-assisted coding tasks
- Generated code often contains critical errors requiring human intervention
- Introduction of Multi-Granularity Debugger (MGDebugger) for hierarchical code debugging
- MGDebugger isolates, identifies, and resolves bugs at different levels of granularity
- Proposed LLM-simulated Python executor to enhance debugging process
- MGDebugger outperforms existing systems with 18.9% accuracy improvement in HumanEval and 97.6% repair success rate in HumanEvalFix
- Demonstrated robustness in fixing bugs across different categories and difficulty levels
- Detailed case study highlights MGDebugger's effectiveness in identifying and correcting buggy parts
- Comprehensive error correction without introducing new bugs by decomposing complex problems into distinct subfunctions
- MGDebugger enhances code clarity and correctness, improving the quality of LLM-generated code

## Summary

Large language models (LLMs) have revolutionized AI-assisted coding tasks, generating code snippets for various programming challenges with impressive proficiency. However, the generated code often contains critical errors that require human intervention to pass tests. This has led to a new paradigm where large models generate code and humans fix it. Debugging LLM-generated code has been a significant challenge. Existing systems treat the erroneous program as a monolithic entity without addressing bugs at varying levels of granularity. To address this issue, this paper introduces Multi-Granularity Debugger (MGDebugger), a hierarchical code debugger that isolates, identifies, and resolves bugs at different levels of granularity. MGDebugger decomposes problematic code into a hierarchical tree structure of subfunctions, each representing a specific level of error granularity. By analyzing and resolving bugs in a bottom-up manner within each subfunction, MGDebugger effectively targets and fixes errors across multiple levels. To enhance the debugging process, an LLM-simulated Python executor is proposed to trace code execution and track variable states accurately. Extensive experiments demonstrate that MGDebugger outperforms existing debugging systems by achieving an 18.9% improvement in accuracy over seed generations in HumanEval and an impressive 97.6% repair success rate in HumanEvalFix. Furthermore, MGDebugger showcases its robustness by effectively fixing bugs across different categories and difficulty levels. A detailed case study illustrates how MGDebugger excels in identifying and correcting buggy parts compared to baseline methods. By decomposing complex problems into distinct subfunctions for separate debugging, MGDebugger ensures comprehensive error correction without introducing new bugs. This approach not only fixes bugs but also enhances code clarity and correctness, showcasing the potential of MGDebugger in improving the quality of LLM-generated code. In conclusion, MGDebugger presents a novel approach to hierarchical code debugging that systematically addresses bugs at multiple levels of granularity. By restructuring complex code into a hierarchical framework and utilizing targeted debugging techniques, MGDebugger demonstrates superior bug-fixing capabilities compared to traditional holistic methods.