# On the Concerns of Developers When Using GitHub Copilot

**Authors:** Xiyu Zhou, Peng Liang, Beiqi Zhang, Zengyang Li, Aakash Ahmad, Mojtaba Shahin, Muhammad Waseem  
**arXiv:** [2311.01020](https://arxiv.org/abs/2311.01020)  
**Date:** 2023-11-02  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2311.01020v1/

## Key points

- Researchers aimed to understand issues and challenges faced by software developers when using GitHub Copilot
- Data was collected from 476 GitHub issues, 706 GitHub discussions, and 184 Stack Overflow posts
- Usage Issue and Compatibility Issue were the most common problems encountered by users
- Main causes of these issues were identified as Copilot Internal Issue, Network Connection Issue, and Editor/IDE Compatibility Issue
- Predominant solutions included Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version
- Researchers established a set of data items for extraction and conducted pilot data extraction on a sample of GitHub issues, discussions, and Stack Overflow posts
- Standards for data extraction were established to ensure accuracy and consistency
- Extracted data items were analyzed using the Constant Comparison method to identify patterns and themes among codes assigned to each item
- Study provides insights into challenges faced by developers when using Copilot in practical development scenarios
- Highlights areas where Copilot can be improved and suggests potential new features desired by users

## Summary

In this study, the researchers aimed to understand the issues and challenges faced by software developers when using GitHub Copilot, an AI-based code generation tool. They collected data from 476 GitHub issues, 706 GitHub discussions, and 184 Stack Overflow posts to identify the common problems, underlying causes, and potential solutions related to Copilot usage. The results showed that Usage Issue and Compatibility Issue were the most common problems encountered by users. The main causes of these issues were identified as Copilot Internal Issue, Network Connection Issue, and Editor/IDE Compatibility Issue. The predominant solutions included Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version. To extract and analyze the data accurately and consistently, the researchers established a set of data items for extraction and conducted a pilot data extraction on a sample of GitHub issues, discussions, and Stack Overflow posts. They also established standards for data extraction to ensure accuracy and consistency. The extracted data items were then analyzed using the Constant Comparison method to identify patterns and themes among the codes assigned to each data item. Overall, this study provides valuable insights into the challenges faced by developers when using Copilot in practical development scenarios. It also highlights areas where Copilot can be improved and suggests potential new features desired by users.