# Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

**Authors:** Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, Mario Fritz  
**arXiv:** [2302.12173](https://arxiv.org/abs/2302.12173)  
**Date:** 2023-02-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.12173v2/

## Key points

- Large Language Models (LLMs) are being integrated into various applications
- LLM functionalities can be modulated via natural language prompts
- Flexibility of LLMs renders them susceptible to targeted adversarial prompting, such as Prompt Injection (PI) attacks
- LLM-integrated applications blur the line between data and instructions, opening up new attack vectors using Indirect Prompt Injection
- Adversaries can remotely exploit LLM-integrated applications by strategically injecting prompts into data likely to be retrieved without direct interface with the user
- A comprehensive taxonomy is derived from a computer security perspective to systematically investigate impacts and vulnerabilities, including data theft, worming, information ecosystem contamination, and other novel security risks
- The study demonstrates the practical viability of these attacks against real-world systems like Bing's GPT-4 powered Chat and code-completion engines as well as synthetic applications built on GPT-4
- Processing retrieved prompts can act as arbitrary code execution, manipulate an application's functionality and control whether or not other APIs are called.
- Effective mitigations for these emerging threats are currently lacking.
- The authors aim to promote safe and responsible deployment of these powerful models while developing robust defenses that protect users and systems from potential attacks.

## Summary

Large Language Models (LLMs) are becoming increasingly integrated into various applications, and their functionalities can be modulated via natural language prompts. However, this flexibility also renders them susceptible to targeted adversarial prompting, such as Prompt Injection (PI) attacks that allow attackers to override original instructions and controls. While it was previously assumed that the user is directly prompting the LLM, a new study argues that LLM-Integrated Applications blur the line between data and instructions, opening up new attack vectors using Indirect Prompt Injection. This allows adversaries to remotely exploit LLM-integrated applications by strategically injecting prompts into data likely to be retrieved without direct interface with the user. The authors derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities, including data theft, worming, information ecosystem contamination, and other novel security risks. They demonstrate the practical viability of these attacks against real-world systems like Bing's GPT-4 powered Chat and code-completion engines as well as synthetic applications built on GPT-4. The study shows how processing retrieved prompts can act as arbitrary code execution, manipulate an application's functionality and control whether or not other APIs are called. Despite increasing integration and reliance on LLMs in various applications, effective mitigations for these emerging threats are currently lacking. By raising awareness of these vulnerabilities and providing key insights into their implications, the authors aim to promote safe and responsible deployment of these powerful models while developing robust defenses that protect users and systems from potential attacks.