# The Technical Debt Dataset

**Authors:** Valentina Lenarduzzi, Nyyti Saarimäki, Davide Taibi  
**arXiv:** [1908.00827](https://arxiv.org/abs/1908.00827)  
**Date:** 2019-08-02  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1908.00827v1/

## Key points

- Technical Debt analysis is increasingly popular in research and industry
- Various tools for static code analysis are used to assess code quality
- Conducting empirical studies on software projects can be costly and time-consuming
- The Technical Debt Dataset provides curated project measurement data from 33 Java projects within the Apache Software Foundation
- The dataset includes information on SonarQube issues, code smells, faults, refactorings, and fault-inducing commits
- PyDriller was used to extract detailed information from Git logs for file modifications in each commit
- Information collected includes software metrics, complexity metrics, test coverage details, and duplications
- The dataset aims to facilitate comparisons in Technical Debt research by providing a common dataset for researchers to utilize

## Summary

Technical Debt analysis is becoming increasingly popular in both research and industry. Various tools for static code analysis are being adopted to assess the quality of code. However, conducting empirical studies on software projects can be costly and time-consuming. Results are often difficult to compare due to differences in projects being considered. To address this issue, the Technical Debt Dataset has been introduced as a curated collection of project measurement data from 33 Java projects within the Apache Software Foundation. In the creation of the Technical Debt Dataset, all commits from defined time frames were analyzed using SonarQube to gather Technical Debt information and Ptidej to detect code smells. Additionally, information on refactoring applied in each commit was collected using Refactoring Miner, along with fault information reported in issue trackers such as Jira. By implementing the SZZ algorithm, fault-inducing and -fixing commits were identified among 78K commits from the selected projects. The dataset revealed significant insights, detecting 1.8M SonarQube issues, 38K code smells, 28K faults, and 57K refactorings across the projects. The extensive project analysis spanned over 200 days and aimed to facilitate comparisons in Technical Debt research by providing a common dataset for researchers to utilize. Furthermore, PyDriller was utilized to extract detailed information from Git logs including file modifications in each commit. Various sets of information related to code quality were collected such as software metrics encompassing size-related metrics, complexity metrics like Cyclomatic Complexity and cognitive complexity, test coverage details including lines not covered by tests, and duplications in terms of duplicated lines and files. Overall, through the integration of multiple tools like PyDriller and Ptidej alongside comprehensive data collection methods, the Technical Debt Dataset offers a valuable resource for researchers looking to delve deeper into technical debt analysis within software projects. The dataset is made accessible through CSV files and an SQLite database for ease of querying data and aims to foster further advancements in understanding technical debt implications on software development practices.