# Defeating Prompt Injections by Design

**Authors:** Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, Florian Tramèr  
**arXiv:** [2503.18813](https://arxiv.org/abs/2503.18813)  
**Date:** 2025-03-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2503.18813v1/

## Key points

- Large Language Models (LLMs) in agentic systems are vulnerable to prompt injection attacks when handling untrusted data
- Authors propose CaMeL as a defense mechanism to create a protective layer around LLMs
- CaMeL extracts control and data flows from trusted queries to prevent untrusted data from impacting program flow
- CaMeL uses capability concept to prevent exfiltration of private data over unauthorized flows
- CaMeL demonstrates effectiveness by solving 67% of tasks with provable security in AgentDojo [NeurIPS 2024]
- Importance of securing both control and data flows against prompt injection attacks in agentic systems is highlighted
- Various defense mechanisms like using delimiters and prompting sandwiching are discussed to make models more resilient to malicious instructions

## Summary

In the paper "Defeating Prompt Injections by Design," authors Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, and Florian Tramèr address the increasing deployment of Large Language Models (LLMs) in agentic systems that interact with external environments. These LLM agents are vulnerable to prompt injection attacks when handling untrusted data. To combat this issue, the authors propose CaMeL, a robust defense mechanism that creates a protective system layer around the LLM to secure it even when underlying models may be susceptible to attacks. CaMeL operates by explicitly extracting control and data flows from trusted queries. This ensures that untrusted data retrieved by the LLM cannot impact program flow. Additionally, CaMeL relies on a capability concept to prevent the exfiltration of private data over unauthorized data flows. The effectiveness of CaMeL is demonstrated through its ability to solve 67% of tasks with provable security in AgentDojo [NeurIPS 2024], an agentic security benchmark. The authors highlight the importance of securing both control and data flows against prompt injection attacks in agentic systems. They discuss various defense mechanisms proposed by researchers to mitigate these risks. Methods such as using delimiters to mark boundaries of untrusted content within context and prompting sandwiching are explored as ways to make models more resilient to malicious instructions. Overall, the paper emphasizes the significance of developing robust defenses like CaMeL to protect LLM agents from prompt injection attacks and ensure secure interactions with external environments in agentic systems.