# How do Machine Learning Projects use Continuous Integration Practices? An Empirical Study on GitHub Actions

**Authors:** João Helis Bernardo, Daniel Alencar da Costa, Sérgio Queiroz de Medeiros, Uirá Kulesza  
**arXiv:** [2403.09547](https://arxiv.org/abs/2403.09547)  
**Date:** 2024-03-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.09547v1/

## Key points

- Continuous Integration (CI) is a well-established practice in traditional software development
- Application of CI practices in Machine Learning (ML) projects is relatively unexplored
- Understanding how CI practices are implemented in ML projects is crucial due to the unique nature of ML development
- Findings from analysis of 185 open-source projects on GitHub show differences in CI adoption between ML and non-ML projects
- ML projects often have longer build durations and lower test coverage compared to non-ML projects, especially medium-sized ones
- Small and medium-sized ML projects exhibit higher prevalence of increasing build duration trends compared to non-ML projects
- Qualitative analysis focused on themes such as CI Build Execution and Status, CI Testing, and CI Infrastructure revealed unique challenges faced by ML projects in adopting CI practices effectively
- Research methodology involved selecting 1,053 relevant ML and non-ML projects with GitHub Actions workflow configuration files from a curated dataset of 4,031 ML projects and 4,076 non-ML projects hosted on GitHub
- Approximately 37% of ML projects have integrated a CI service into their workflow for tasks like software testing and building, but there is a knowledge gap regarding adoption patterns of CI practices within ML projects
- Study aims to provide valuable insights for developing customized approaches for effectively adopting CI practices within the unique domain of Machine Learning

## Summary

Continuous Integration (CI) is a well-established practice in traditional software development. However, its application in Machine Learning (ML) projects remains relatively unexplored. Understanding how CI practices are implemented in ML projects is crucial for tailoring effective approaches due to the unique nature of ML development. In this study, we conducted a comprehensive analysis of 185 open-source projects on GitHub to uncover differences in CI adoption between ML and non-ML projects. Our findings revealed that ML projects often have longer build durations and medium-sized ML projects exhibit lower test coverage compared to their non-ML counterparts. Additionally, small and medium-sized ML projects showed a higher prevalence of increasing build duration trends compared to non-ML projects. Through qualitative analysis, we explored discussions around CI in both ML and non-ML projects, focusing on themes such as CI Build Execution and Status, CI Testing, and CI Infrastructure. These insights shed light on the unique challenges faced by ML projects in effectively adopting CI practices. In our research methodology, we explained how we selected the studied projects from a curated dataset of 4,031 ML projects and 4,076 non-ML projects hosted on GitHub. By applying filters to the initial dataset, we narrowed down our selection to 1,053 projects with GitHub Actions workflow configuration files. This process allowed us to delve deeper into analyzing data from a subset of relevant ML and non-ML projects. Previous studies have highlighted the importance of CI in enabling more frequent software releases and emphasized specific CI practices employed by software projects beyond just adopting CI services. In the context of ML projects, approximately 37% have integrated a CI service into their workflow with common tasks including software testing and building. However, there remains a knowledge gap regarding adoption patterns of CI practices within ML projects. By addressing research questions related to how CI is applied and discussed by developers in the ML development domain, our study aims to contribute valuable insights that can guide the development of customized approaches for effectively adopting CI practices within this unique domain. Through our detailed analysis of project data and discussions surrounding CI practices in both ML and non-ML contexts, we aim to provide a nuanced understanding of the challenges and opportunities associated with implementing continuous integration in Machine Learning projects.