# RAFT: Adapting Language Model to Domain Specific RAG

**Authors:** Tianjun Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, Joseph E. Gonzalez  
**arXiv:** [2403.10131](https://arxiv.org/abs/2403.10131)  
**Date:** 2024-03-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.10131v2/

## Key points

- Pretraining is a standard practice in large language models (LLMs) to incorporate vast amounts of textual data.
- RAFT is introduced as a novel training recipe to enhance LLMs' ability to answer questions in an "open-book" in-domain setting.
- RAFT trains the model to disregard distractor documents and focus on citing relevant sequences from retrieved documents verbatim, improving reasoning capabilities with chain-of-thought-style responses.
- RAFT is tailored for domain-specific RAG tasks and consistently boosts performance across datasets like PubMed, HotpotQA, and Gorilla.
- The code and demo for RAFT are openly available on GitHub at github.com/ShishirPatil/gorilla.
- Related works include concepts like Retrieval-Augmented Language Models (RALMs) and fine-tuning pretrained LLMs specifically for RAG tasks.
- RAFT prioritizes testing LLMs on consistent sets of documents compared to other works that adapt models for varied domains at test time.
- In conclusion, RAFT offers a strategic approach to improving LLM performance in domain-specific question-answering tasks within an "open-book" context, demonstrating significant potential for real-world applications.

## Summary

In the realm of large language models (LLMs), pretraining has become a standard practice for incorporating vast amounts of textual data. However, the most effective approach for integrating new knowledge into these models remains an open question. To address this challenge, this paper introduces RAFT - a novel training recipe that enhances LLMs' ability to answer questions in an "open-book" in-domain setting. By training the model to disregard distractor documents and focus on citing relevant sequences from retrieved documents verbatim, RAFT improves reasoning capabilities with its chain-of-thought-style responses. This approach is specifically tailored for domain-specific RAG tasks and consistently boosts performance across datasets such as PubMed, HotpotQA, and Gorilla. The code and demo for RAFT are openly available on GitHub at github.com/ShishirPatil/gorilla. Related works in the field have explored concepts like Retrieval-Augmented Language Models (RALMs) and memorization in neural language models. Recent studies have also focused on fine-tuning pretrained LLMs specifically for RAG tasks. While these works adapt models for varied domains at test time, RAFT stands out by prioritizing testing LLMs on consistent sets of documents. In conclusion, RAFT presents a strategic approach to bolstering a model's performance in domain-specific question-answering tasks within an "open-book" context. Through meticulous design decisions and evaluations across diverse datasets, RAFT demonstrates significant potential for improving LLM capabilities in real-world applications.