# Prompt Stealing Attacks Against Large Language Models

**Authors:** Zeyang Sha, Yang Zhang  
**arXiv:** [2402.12959](https://arxiv.org/abs/2402.12959)  
**Date:** 2024-02-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.12959v1/

## Key points

- Increasing reliance on large language models (LLMs) like ChatGPT highlights the importance of prompt engineering
- Novel attack strategy: prompt stealing attacks aim to steal well-crafted prompts by analyzing generated answers
- Comprises two key modules: parameter extractor and prompt reconstructor
- Parameter extractor categorizes original prompts into direct, role-based, or in-context prompts based on responses
- Prompt reconstructor reconstructs stolen prompts based on extracted features and generated answers
- Defense strategies against prompt stealing attacks involve a trade-off between attack similarity and utility
- Existing methods show promise but may reduce utility; need for more automated defense mechanisms
- Continued research and innovation are essential to enhance defenses against evolving threats targeting LLMs

## Summary

The increasing reliance on large language models (LLMs) like ChatGPT in various fields underscores the significance of prompt engineering. This technology aims to enhance the quality of model outputs and has become a compelling challenge as companies invest heavily in expert prompt engineers and educational resources to meet market demand. In response to this landscape, a novel attack strategy against LLMs has been introduced: prompt stealing attacks. Prompt stealing attacks are designed to pilfer well-crafted prompts by analyzing generated answers. This attack comprises two key modules: the parameter extractor and the prompt reconstructor. The parameter extractor identifies the characteristics of original prompts by categorizing them into direct prompts, role-based prompts, or in-context prompts based on generated responses. It then predicts specific roles or contexts used in these prompts. Following this extraction process, the prompt reconstructor reconstructs stolen prompts based on extracted features and generated answers with the goal of producing reversed prompts similar to the originals. Experimental results have demonstrated the efficacy of these proposed attacks, shedding light on a new dimension within prompt engineering and highlighting security concerns surrounding LLMs. Additionally, defense strategies against prompt stealing attacks have been explored, revealing a trade-off between attack similarity and utility. While existing defense methods show promise in mitigating risks associated with such attacks, they may also lead to a reduction in utility. Therefore, there is a need for more automated defense mechanisms that can effectively safeguard against these threats without imposing significant operational burdens on users. In conclusion, while current defense strategies offer some level of protection against prompt stealing attacks, there is still room for improvement to achieve an optimal trade-off between security and utility. Continued research and innovation in this area are essential to bolster defenses against evolving threats targeting LLMs.