# Code Evolution Graphs: Understanding Large Language Model Driven Design of Algorithms

**Authors:** Niki van Stein, Anna V. Kononova, Lars Kotthoff, Thomas Bäck  
**arXiv:** [2503.16668](https://arxiv.org/abs/2503.16668)  
**Date:** 2025-03-20  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2503.16668v1/

## Key points

- Large Language Models (LLMs) are used to generate code in evolutionary computation frameworks for algorithm optimization
- Challenges arise when generated algorithms are not competitive or optimization stalls due to a lack of understanding of the generation process
- A novel approach has been proposed to analyze the generated code during the evolutionary process using Abstract Syntax Trees (ASTs)
- Metrics and features extracted from ASTs include structural properties, graph centrality metrics, clustering coefficients, assortativity, entropy measures, and code complexity features
- These features provide insights into algorithmic structures, diversity, scalability, maintainability, computational efficiency, and trade-offs between algorithmic sophistication and runtime performance
- LLMs tend to generate more complex code with repeated prompting but excessive complexity can hinder algorithmic performance in certain cases
- Different LLMs exhibit distinct coding styles which suggests using multiple LLMs within code evolution frameworks may lead to higher-performing algorithms compared to relying on a single LLM

## Summary

, , , , 

Large Language Models (LLMs) have shown promise in generating code within evolutionary computation frameworks to optimize algorithms. However, challenges arise when the generated algorithms are not competitive or optimization stalls due to a lack of understanding of the generation process and resulting code. To address this issue, a novel approach has been proposed to enable users to analyze the generated code during the evolutionary process and observe how it evolves with repeated prompting of the LLM. The methodology for analyzing the evolution of generated code involves leveraging Abstract Syntax Trees (ASTs) as a foundational representation of the code. Various metrics and features are extracted from ASTs, including structural properties metrics such as node count and edge count, graph centrality metrics like eigenvector centrality, clustering coefficients, transitivity, assortativity, and entropy measures. These features provide insights into algorithmic structures and diversity, aiding in visualizing code evolution. Additionally, code complexity features are utilized to understand the scalability, maintainability, and computational efficiency of generated code. Metrics such as cyclomatic complexity, token count, parameter count, function-level aggregates for various complexity measures, and depth/nesting metrics are employed to assess trade-offs between algorithmic sophistication and runtime performance. The comprehensive set of metrics derived from ASTs and code complexity analysis allows for a comparative analysis of algorithmic structures and diversity. This approach sheds light on how LLMs tend to generate more complex code with repeated prompting but also highlights that excessive complexity can hinder algorithmic performance in certain cases. Furthermore, different LLMs exhibit distinct coding styles, suggesting that using multiple LLMs within code evolution frameworks may lead to higher-performing algorithms compared to relying on a single LLM. Overall,<kgd> this refined methodology provides valuable insights into the properties of generated algorithms that impact performance.