# Adversarial Attacks on Image Generation With Made-Up Words

**Authors:** Raphaël Millière  
**arXiv:** [2208.04135](https://arxiv.org/abs/2208.04135)  
**Date:** 2022-08-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2208.04135v1/

## Key points

- Text-guided image generation models can generate images using nonce words to evoke specific visual concepts
- Two approaches for prompting: macaronic prompting and evocative prompting
- Macaronic prompting involves creating cryptic hybrid words from different languages
- Evocative prompting involves designing nonce words with morphological features similar to existing words
- These two methods can be combined for more specific visual concepts
- Text-guided image generation models are vulnerable to adversarial attacks
- Vulnerability may vary based on factors such as model size, architecture, tokenization procedure, and training data
- Further research is needed to understand how different models respond to macaronic and evocative prompting attacks
- Concerns about circumventing content moderation and generating offensive or harmful images exist
- Adversarial attacks have been explored in the context of vision-language models for image captioning and recognition
- Mitigation strategies need to be developed to counter malicious use of text-guided image generation technology

## Summary

Text-guided image generation models can be prompted to generate images using nonce words that are designed to evoke specific visual concepts. This can be achieved through two approaches: macaronic prompting and evocative prompting. Macaronic prompting involves creating cryptic hybrid words by combining subword units from different languages, while evocative prompting involves designing nonce words with morphological features similar to existing words to trigger visual associations. These two methods can also be combined to generate images associated with more specific visual concepts. However, it is important to note that text-guided image generation models are not immune to adversarial attacks. The vulnerability of these models to text-based adversarial attacks may vary depending on factors such as model size, architecture, tokenization procedure, and training data. While some attacks may work reliably across different models, further research is needed to understand the factors that determine how different models respond to macaronic and evocative prompting. One significant concern raised by these techniques is their potential for circumventing existing approaches to content moderation. There is a risk of generating offensive or harmful images using these methods. Adversarial attacks on text-guided image generation models have been explored in the context of vision-language models for image captioning and recognition as well. For example, typographic attacks involve applying real-life erroneous labels to items in an image, causing vision-language models to misclassify them. To mitigate the malicious use of these techniques for generating harmful or offensive visual content, effective strategies need to be developed. Further research is necessary not only to understand how different models respond to adversarial prompting but also explore ways of countering such attacks and ensuring responsible use of text-guided image generation technology.