# Making Retrieval-Augmented Language Models Robust to Irrelevant Context

**Authors:** Ori Yoran, Tomer Wolfson, Ori Ram, Jonathan Berant  
**arXiv:** [2310.01558](https://arxiv.org/abs/2310.01558)  
**Date:** 2023-10-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.01558v1/

## Key points

- Retrieval-augmented language models (RALMs) in Natural Language Processing are effective in creating factual, efficient, and up-to-date systems.
- The key challenge for RALMs is to ensure that retrieved information enhances model performance when relevant and does not hinder it when irrelevant, especially in scenarios requiring multi-hop reasoning.
- Recent studies have shown instances where retrieval augmentation can decrease performance, leading to errors cascading through the system.
- Two methods were proposed to address this challenge: a baseline approach filtering out irrelevant passages using a natural language inference (NLI) model, and a novel approach generating training data for fine-tuning language models with both relevant and irrelevant contexts.
- Empirical results demonstrated that training models on just 1,000 examples could help them handle irrelevant contexts robustly while maintaining high performance on relevant ones.
- Efforts were made towards developing Large Language Models (LLMs) with controllable memory capabilities to ignore irrelevant context by training on smaller sets of questions and automatically generated data.
- Simple NLI models were found effective in increasing robustness against irrelevant context at the cost of discarding some relevant passages when training data is limited.

## Summary

In the field of Natural Language Processing, retrieval-augmented language models (RALMs) have shown promise in creating systems that are factual, efficient, and up-to-date. The key challenge for RALMs is to ensure that retrieved information enhances model performance when relevant and does not hinder it when irrelevant. This becomes crucial in scenarios requiring multi-hop reasoning, where the misuse of irrelevant evidence can lead to errors cascading through the system. Recent studies have highlighted instances where retrieval augmentation can actually decrease performance. In response to this issue, a comprehensive analysis was conducted on five open-domain question answering benchmarks to identify cases where retrieval negatively impacts accuracy. Two methods were proposed to address this challenge: firstly, a baseline approach that filters out retrieved passages not supported by question-answer pairs according to a natural language inference (NLI) model. While effective in preventing performance reduction, this method also risks discarding relevant passages. To overcome this limitation, a novel approach was introduced to automatically generate training data for fine-tuning language models to effectively utilize retrieved passages by exposing them to a mix of relevant and irrelevant contexts during training. Empirical results demonstrated that even with just 1,000 examples, the model could be trained to handle irrelevant contexts robustly while maintaining high performance on relevant ones. Additionally, efforts were made towards developing Large Language Models (LLMs) with controllable memory capabilities that enable them to ignore irrelevant context. Unlike previous approaches which relied on over 200K training examples, the focus here was on training with a smaller set of questions and automatically generated data. The study also emphasized multi-hop question-answering settings where retrievers are utilized multiple times. In conclusion, the research highlighted the importance of making RALMs robust against irrelevant retrieved context to enhance overall performance in various tasks. Simple NLI models were found effective in increasing robustness at the cost of discarding some relevant passages when training data is limited. By training models on as few as 1,000 examples and exposing them to diverse contexts during training, significant improvements in handling irrelevant information were observed while maintaining high performance levels overall.