# Jailbreaking Black Box Large Language Models in Twenty Queries

**Authors:** Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, Eric Wong  
**arXiv:** [2310.08419](https://arxiv.org/abs/2310.08419)  
**Date:** 2023-10-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.08419v1/

## Key points

- Growing interest in ensuring Large Language Models (LLMs) align with human values
- Vulnerabilities in LLM alignment process can lead to adversarial jailbreaks
- Introduction of Prompt Automatic Iterative Refinement (PAIR) algorithm to address vulnerabilities
- PAIR generates semantic jailbreaks using social engineering tactics and black-box access to LLM
- System prompt for PAIR guides attacker LLM to act as red team against target LLM, emphasizing role-playing and emotional manipulation
- PAIR iteratively refines candidate jailbreaks based on previous prompts, responses, and scores
- Experimental evaluations show PAIR's efficacy in jailbreaking open-source and closed-source LLMs with competitive success rates
- Challenges in evaluating jailbreaking attacks due to complexity of generating semantically rich content
- Proposal of JUDGE function parameterized by an LLM to assess candidate jailbreaking prompts based on creativity and semantics
- Strategies like PAIR offer insights into enhancing model robustness against adversarial attacks while promoting ethical alignment with human values

## Summary

In the realm of Large Language Models (LLMs), there is a growing interest in ensuring that these models align with human values. The alignment process is susceptible to adversarial jailbreaks, where LLMs can be coerced into bypassing their safety mechanisms. Identifying and understanding these vulnerabilities are crucial in preventing potential misuse of LLMs. To address this issue, a new algorithm called Prompt Automatic Iterative Refinement (PAIR) has been proposed. PAIR operates by generating semantic jailbreaks using only black-box access to an LLM, inspired by social engineering tactics. The system prompt for PAIR guides the attacker LLM to act as a red team against the target LLM, emphasizing the use of social engineering techniques like role-playing and emotional manipulation in crafting jailbreaking prompts. The algorithm iteratively refines candidate jailbreaks by analyzing previous prompts, responses, and scores to identify areas for improvement. In experimental evaluations using the "harmful behaviors" subset of the AdvBench benchmark, PAIR demonstrated its efficacy in jailbreaking both open-source LLMs like Vicuna-13B-v1.5 and closed-source LLMs such as GPT-3.5 and 4. By curating a representative subset of harmful behavior objectives and utilizing deterministic generation with specific temperature settings, PAIR achieved competitive success rates in producing jailbreak prompts with minimal queries. Evaluating the performance of jailbreaking attacks poses challenges due to the complexity of generating semantically rich content. To address this issue, a JUDGE function parameterized by an LLM was proposed to assess candidate jailbreaking prompts based on creativity and semantics. This approach leverages system prompts to instruct the LLM to score responses on a scale from 1 to 10, indicating the level of detail in a fully jailbroken response. As language models continue to advance, strategies like PAIR offer insights into enhancing model robustness against adversarial attacks while promoting ethical alignment with human values. Through iterative refinement processes guided by system prompts and automated generation techniques, PAIR showcases promising results in efficiently identifying vulnerabilities within LLMs for improved security measures.