# RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation

**Authors:** Shi-Qi Yan, Quan Liu, Zhen-Hua Ling  
**arXiv:** [2501.13726](https://arxiv.org/abs/2501.13726)  
**Date:** 2025-01-23  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2501.13726v2/

## Key points

- Retrieval-Augmented Generation (RAG) enhances response generation by leveraging external knowledge
- Dependency on the quality and accuracy of retrieved context is a key challenge for RAG
- Retrieval Preference Optimization (RPO) is introduced to address this limitation
- RPO dynamically leverages multi-source knowledge based on retrieval relevance, integrating it into the reward model
- RPO surpasses RAG by 4-10% in accuracy across four datasets without requiring additional components

## Summary

, , , , 

In the realm of natural language processing, Retrieval-Augmented Generation (RAG) has shown promise in harnessing external knowledge to enhance response generation. However, a key challenge lies in the dependency of RAG's generation process on the quality and accuracy of the retrieved context. This issue is particularly pronounced when large language models (LLMs) are tasked with evaluating non-parametric knowledge retrieved from external sources that may conflict with their internal memorization. To address this limitation, a novel approach called Retrieval Preference Optimization (RPO) has been introduced. RPO serves as a lightweight yet effective alignment method designed to dynamically leverage multi-source knowledge based on retrieval relevance. By deriving an implicit representation of retrieval relevance and integrating it into the reward model, RPO seamlessly combines retrieval evaluation and response generation within a single framework. This integration eliminates the need for additional procedures to assess retrieval quality, distinguishing RPO as a unique alignment approach dedicated to enhancing RAG. One notable aspect of RPO is its ability to quantify the awareness of retrieval relevance during training, thereby overcoming mathematical obstacles that hinder previous methods. Experimental results conducted across four datasets demonstrate that RPO surpasses RAG by 4-10% in accuracy without requiring any supplementary components. This performance improvement underscores RPO's robust generalization capabilities and solidifies its position as a valuable advancement in optimizing retrieval-augmented generation processes. Authored by Shi-Qi Yan, Quan Liu, and Zhen-Hua Ling, the research paper titled "RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation" delves into these innovative techniques and their implications for enhancing natural language processing tasks.