# RAG4Tickets: AI-Powered Ticket Resolution via Retrieval-Augmented Generation on JIRA and GitHub Data

**Authors:** Mohammad Baqar  
**arXiv:** [2510.08667](https://arxiv.org/abs/2510.08667)  
**Date:** 2025-10-09  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2510.08667v1/

## Key points

- Challenges faced by software teams in resolving recurring or related issues due to fragmented knowledge across platforms like JIRA tickets, developer discussions, and GitHub pull requests
- Introduction of the Retrieval-Augmented Generation (RAG) framework integrating Sentence-Transformers with FAISS-based vector search for context-aware ticket resolution recommendations
- Utilization of historical JIRA tickets, user comments, and linked PR metadata to retrieve semantically similar past cases and generate explainable resolution suggestions using a Large Language Model (LLM)
- Aim of the framework to improve resolution accuracy, fix quality, and knowledge reuse in modern DevOps environments through a unified pipeline linking JIRA and GitHub data
- Challenges in implementing the RAG framework such as unreliable suggestions from LLM, historical data bias, dataset drift, lack of explainability, scalability issues, and cost overhead
- Proposed solutions for overcoming challenges including retrieval grounding through evidence-weighted decoding, temporal relevance weighting for dataset drift, explainability enhancements, periodic re-indexing and embedding refresh
- Future work suggestions like adaptive learning pipelines with online embedding updates for scalability, model distillation for new domains, incremental fine-tuning driven by recent commits for evolving enterprise data

## Summary

In modern software development environments, software teams often face challenges in resolving recurring or related issues due to fragmented knowledge scattered across various platforms such as JIRA tickets, developer discussions, and GitHub pull requests. To address this issue, a Retrieval-Augmented Generation (RAG) framework has been proposed. This framework integrates Sentence-Transformers for semantic embeddings with FAISS-based vector search to provide context-aware ticket resolution recommendations. The approach involves embedding historical JIRA tickets, user comments, and linked PR metadata to retrieve semantically similar past cases. These retrieved cases are then synthesized by a Large Language Model (LLM) into grounded and explainable resolution suggestions. The framework aims to improve resolution accuracy, fix quality, and knowledge reuse in modern DevOps environments through a unified pipeline linking JIRA and GitHub data. : However, there are challenges that need to be addressed for the successful implementation of the RAG framework. These include issues such as unreliable or misleading suggestions generated by the LLM not grounded in retrieved evidence, historical data bias from outdated or low-quality fixes in legacy tickets and PRs, dataset drift due to changes in frameworks reducing relevance of older examples and embeddings, lack of explainability leading to developer hesitation in adopting AI recommendations without transparent supporting evidence, scalability and cost overhead from indexing large-scale repositories and frequent LLM inference increasing compute and operational costs. : To overcome these challenges,such as retrieval grounding through evidence-weighted decoding,, , , < kd>temporal relevance weighting</kd> for dataset drift;  for explainability; periodic re-indexing and embedding refresh for knowledge obsolescence have been suggested. : The future work includes adaptive learning pipelines with online embedding updates for scalability; model distillation for new domains; incremental fine-tuning driven by recent commits for evolving enterprise data. Overall, the RAG framework demonstrates how retrieval and generation can be effectively fused to enhance AI-assisted ticket triage and resolution generation processes in DevOps workflows.