# Exploring LLM-based Agents for Root Cause Analysis

**Authors:** Devjeet Roy, Xuchao Zhang, Rashi Bhave, Chetan Bansal, Pedro Las-Casas, Rodrigo Fonseca, Saravan Rajmohan  
**arXiv:** [2403.04123](https://arxiv.org/abs/2403.04123)  
**Date:** 2024-03-07  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2403.04123v1/

## Key points

- Roy et al. presented a study on incident management within cloud-based software systems at Conference’17 in July 2017 in Washington, DC, USA.
- Incident management plays a crucial role in the software development lifecycle due to the increasing complexity of these systems.
- Root cause analysis (RCA) is identified as a key aspect of incident management that requires specialized knowledge and experience from on-call engineers.
- Manual RCA is time-consuming and burdensome for engineers, prompting Roy et al. to propose automation as a solution to streamline the process.
- The ReAct agent introduced by Roy et al. utilizes retrieval tools to enhance the diagnostic capabilities of Large Language Models (LLMs) for RCA.
- Empirical evaluation using real-world production incidents from Microsoft shows that ReAct performs competitively with strong retrieval and reasoning baselines while improving factual accuracy significantly.
- Discussions associated with incident reports do not yield substantial performance enhancements according to semantic metrics.
- A case study conducted with a team at Microsoft demonstrates how integrating external diagnostic services can overcome limitations observed in prior studies and provide practical insights for implementing such systems in operational settings.

## Summary

At Conference’17 in July 2017 in Washington, DC, USA, Roy et al. presented their study on incident management within cloud-based software systems. They highlight the increasing complexity of these systems and the crucial role of incident management in the software development lifecycle. The authors identify root cause analysis (RCA) as a key aspect of incident management that requires specialized knowledge and experience from on-call engineers. However, manual RCA is time-consuming and can burden engineers. To address this issue, Roy et al. propose automation as a solution to streamline the process and alleviate the burden on engineers. Building upon previous research that utilized Large Language Models (LLMs) for RCA, Roy et al. introduce their ReAct agent equipped with retrieval tools to enhance the diagnostic capabilities of LLMs. Through an empirical evaluation using real-world production incidents from Microsoft, they demonstrate that ReAct performs competitively with strong retrieval and reasoning baselines while significantly improving factual accuracy. Additionally, the incorporation of discussions associated with incident reports does not yield substantial performance enhancements according to semantic metrics. Furthermore, Roy et al. conduct a case study with a team at Microsoft to equip the ReAct agent with access to external diagnostic services used for manual RCA. This integration showcases how agents can overcome limitations observed in prior studies and provides practical insights for implementing such systems in operational settings. Through detailed evaluations and qualitative analyses, Roy et al. 's work contributes valuable insights into enhancing RCA processes through automated agents within cloud-based software systems.