# The Matthew Effect of AI Programming Assistants: A Hidden Bias in Software Evolution

**Authors:** Fei Gu, Zi Liang, Hongzong LI, Jiahao MA  
**arXiv:** [2509.23261](https://arxiv.org/abs/2509.23261)  
**Date:** 2025-09-27  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2509.23261v1/

## Key points

- The impact of AI-assisted programming on software development practices is explored
- Large language models (LLMs) influence coding paradigms through concepts like vibe coding and agentic coding
- Extensive experiments conducted on algorithmic programming tasks and framework selection tasks to analyze the interaction between AI-driven programming and the software ecosystem
- Discovery of a significant Matthew effect where LLM-generated code's success rate is correlated with the popularity of the programming language or framework being used, potentially reinforcing existing hierarchies
- Detailed experimental infrastructure outlined for conducting tasks using proprietary LLM APIs and three specific AI programming tools
- Methodology involves standardizing prompts across different technologies while maintaining consistent functional requirements
- In-depth analysis of AI coding processes, including cleaning up non-executable content from generated responses to ensure only functional code remains
- Approach to judging solutions generated by AI assistants using platforms like LeetCode highlighted
- Study provides insights into hidden biases present in AI-driven programming assistance at both language and framework levels, opening discussions about their impact on software ecosystem trajectories, innovation, and diversity in software development practices.

## Summary

The authors of this paper explore the impact of AI-assisted programming on software development practices. They discuss the emergence of large language models (LLMs) and their influence on coding paradigms through concepts like vibe coding and agentic coding. The study conducts extensive experiments on algorithmic programming tasks and framework selection tasks to analyze the interaction between AI-driven programming and the software ecosystem. One notable finding is the discovery of a significant Matthew effect, where LLM-generated code's success rate is directly correlated with the popularity of the programming language or framework being used. This highlights how AI systems may reinforce existing hierarchies and potentially stifle diversity and innovation within the programming community. The paper also outlines a detailed experimental infrastructure for conducting these tasks using proprietary LLM APIs and three specific AI programming tools. The methodology involves standardizing prompts across different technologies while maintaining consistent functional requirements. The authors also delve into an in-depth analysis of AI coding processes, discussing how they clean up non-executable content from generated responses to ensure only functional code remains. Additionally, they explain their approach to judging solutions generated by AI assistants using platforms like LeetCode. Overall, this comprehensive study provides valuable insights into hidden biases present in AI-driven programming assistance at both language and framework levels. By shedding light on these structural biases, it opens up discussions about how they shape software ecosystem trajectories and influence factors like innovation and diversity in software development practices.