# SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

**Authors:** John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press  
**arXiv:** [2405.15793](https://arxiv.org/abs/2405.15793)  
**Date:** 2024-05-06  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2405.15793v1/

## Key points

- Introduction of SWE-agent as an autonomous system utilizing a language model for computer interaction in software engineering tasks
- Implementation of a custom-built agent-computer interface (ACI) enhancing capabilities such as code file creation, modification, repository navigation, and program execution
- Impressive performance on the SWE-bench platform with 12.5% issue resolution, surpassing previous achievements with retrieval-augmented generation (RAG)
- Impact of ACI design on agent behavior and performance, providing insights on effective design strategies
- Evolution of software engineering benchmarks to evaluate language model performance through diverse challenges like translating problems into different languages, incorporating libraries, and enhancing test coverage
- Use of software engineering as an evaluation domain for language models by integrating real-world SE subtasks like automated program repair, bug localization, and testing
- Leveraging the SWE-bench dataset for evaluation with rigorous automatic execution-based methods showcasing the efficacy of the approach
- Potential of language models like SWE-agent in addressing complex software engineering challenges and the importance of thoughtful ACI design for optimizing agent performance

## Summary

, , , , 

In the realm of software engineering, the task of code generation and computer interaction is a complex challenge that requires a high level of proficiency. In this paper, we introduce SWE-agent, an autonomous system that utilizes a language model to effectively interact with computers in order to tackle various software engineering tasks. Through the implementation of a custom-built agent-computer interface (ACI), we demonstrate how this interface significantly enhances the agent's capabilities to create and modify code files, navigate through entire repositories, and execute programs. On the SWE-bench platform, our SWE-agent showcases impressive performance by successfully resolving 12.5% of issues, surpassing the previous best achievement of 3.8% with retrieval-augmented generation (RAG). We delve into the impact of ACI design on the behavior and overall performance of the agent, offering valuable insights on effective design strategies. Furthermore, we explore related work in software engineering benchmarks, highlighting advancements in code generation tasks that evaluate language model performance. These benchmarks have evolved to encompass diverse challenges such as translating problems into different programming languages, incorporating third-party libraries, and enhancing test coverage. Additionally, recent efforts have emphasized using software engineering as a robust evaluation testbed for language models by integrating real-world SE subtasks like automated program repair, bug localization, and testing within a unified task formulation. By leveraging the comprehensive SWE-bench dataset comprising task instances from various GitHub repositories and employing rigorous automatic execution-based evaluation methods, our study underscores the significance of utilizing software engineering as a multifaceted evaluation domain for language models. Through detailed experimental setups and analysis on both full-scale SWE-bench test sets and focused subsets like SWE-bench Lite for functional bug fixes evaluation, we present compelling results that underscore the efficacy of our approach. In conclusion, our research sheds light on the potential of language models as agents in tackling intricate software engineering challenges and emphasizes the importance of thoughtful ACI design in optimizing agent performance. The integration of cutting-edge technologies like SWE-agent opens up new avenues for advancing automation in software development processes while showcasing promising outcomes in enhancing code generation capabilities and repository-level code editing tasks.