# PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

**Authors:** Wei Zou, Runpeng Geng, Binghui Wang, Jinyuan Jia  
**arXiv:** [2402.07867](https://arxiv.org/abs/2402.07867)  
**Date:** 2024-02-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.07867v3/

## Key points

- Large language models (LLMs) have impressive generative capabilities but also limitations such as outdated knowledge and potential hallucination.
- Retrieval-Augmented Generation (RAG) leverages external knowledge from databases to enhance answer generation.
- PoisonedRAG is a new attack surface targeting LLMs by injecting malicious texts into the knowledge database to manipulate generated answers.
- The authors frame knowledge corruption attacks as an optimization problem and present solutions tailored to different attacker backgrounds.
- PoisonedRAG can achieve a high success rate of 90% by injecting just five malicious texts per target question into a large knowledge database.
- Existing defense mechanisms against PoisonedRAG are found inadequate, highlighting the need for novel defense strategies in securing RAG systems.

## Summary

, , , , 

Large language models (LLMs) have revolutionized natural language processing with their impressive generative capabilities, but they also come with limitations such as outdated knowledge and potential hallucination. To address these issues, Retrieval-Augmented Generation (RAG) has emerged as a cutting-edge technique that leverages external knowledge from databases to enhance answer generation. In this work by Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia, the authors delve into the security implications of RAG systems. They identify a new attack surface introduced by the knowledge database in RAG systems and propose PoisonedRAG as the first knowledge corruption attack targeting LLMs. In PoisonedRAG, attackers can inject malicious texts into the knowledge database to manipulate LLMs into generating specific answers for chosen questions. The authors frame knowledge corruption attacks as an optimization problem where the goal is to identify a set of malicious texts that can successfully induce targeted responses from LLMs. They consider different attacker backgrounds, including black-box and white-box settings, and present two solutions tailored to each scenario. Through their experiments, the researchers demonstrate that PoisonedRAG can achieve a high success rate of 90% by injecting just five malicious texts per target question into a large knowledge database. Furthermore, they evaluate existing defense mechanisms against PoisonedRAG and find them inadequate in thwarting such attacks, underscoring the urgent need for novel defense strategies in securing RAG systems. This study sheds light on the vulnerabilities inherent in RAG systems and underscores the importance of considering security implications alongside performance enhancements in developing advanced language models. The findings presented here pave the way for future research efforts aimed at fortifying RAG systems against malicious manipulation while preserving their innovative capabilities in natural language generation.