# Defending Against Indirect Prompt Injection Attacks With Spotlighting

**Authors:** Keegan Hines, Gary Lopez, Matthew Hall, Federico Zarfati, Yonatan Zunger, Emre Kiciman  
**arXiv:** [2403.14720](https://arxiv.org/abs/2403.14720)  
**Date:** 2024-03-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.14720v1/

## Key points

- Large Language Models (LLMs) are powerful tools in natural language processing that have revolutionized the field.
- LLMs are vulnerable to indirect prompt injection attacks where adversarial instructions can be embedded into untrusted data processed alongside user commands.
- Researchers have introduced spotlighting techniques as a form of prompt engineering to improve LLMs' ability to distinguish among multiple sources of input.
- Spotlighting involves using encoding algorithms such as base64, ROT13, or binary transformations on the input text to enhance the model's ability to differentiate between user commands and potentially malicious instructions.
- Experimental methodology has shown that spotlighting is an effective defense against indirect prompt injection attacks when applied to models like GPT-3.5Turbo and GPT-4 from the GPT family, reducing attack success rate from over 50% to below 2% without impacting task efficacy significantly.
- There is potential for further research into multi-channel analogs for LLMs inspired by out-of-band signaling methods used in telecommunications, which could involve passing control tokens separately from data tokens to enhance security measures.
- Techniques like delimiting, marking, and encoding transformations provide a robust defense mechanism against adversarial instructions while maintaining system functionality.

## Summary

Large Language Models (LLMs) are powerful tools in natural language processing that have revolutionized the field. However, they are also vulnerable to indirect prompt injection attacks where adversarial instructions can be embedded into untrusted data processed alongside user commands. To address this vulnerability, researchers have introduced spotlighting techniques as a form of prompt engineering to improve LLMs' ability to distinguish among multiple sources of input. One approach to spotlighting involves using encoding algorithms such as base64, ROT13, or binary transformations on the input text. By transforming the text in a recognizable way, the model can more easily identify the provenance of each section of input. This method enhances the model's ability to differentiate between user commands and potentially malicious instructions embedded in the data. Experimental methodology has shown that spotlighting is an effective defense against indirect prompt injection attacks when applied to models like GPT-3.5Turbo and GPT-4 from the GPT family. By implementing spotlighting techniques, the attack success rate can be reduced from over 50% to below 2% without significantly impacting task efficacy. Looking ahead, there is potential for further research into multi-channel analogs for LLMs inspired by out-of-band signaling methods used in telecommunications. This approach could involve passing control tokens separately from data tokens to ensure that the model only reacts to instructive tokens from a control layer. While current architectures may not support this concept directly, it presents an intriguing avenue for future exploration and development in enhancing LLM security measures. In conclusion,  offers a promising solution to mitigate indirect prompt injection attacks on large language models by making input provenance more salient while maintaining semantic content and task performance. Through techniques like delimiting, marking, and encoding transformations, provides a robust defense mechanism against adversarial instructions without compromising overall system functionality.