# Adversarial YOLO: Defense Human Detection Patch Attacks via Detecting Adversarial Patches

**Authors:** Nan Ji, YanFei Feng, Haidong Xie, Xueshuang Xiang, Naijin Liu  
**arXiv:** [2103.08860](https://arxiv.org/abs/2103.08860)  
**Date:** 2021-03-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2103.08860v1/

## Key points

- Security of object detection systems is a major concern due to adversarial patch attacks
- Existing defense mechanisms struggle to defend against attacks on human detection systems
- Ad-YOLO is a new defense strategy specifically targeted at human detection attacks
- Ad-YOLO is an efficient and effective plug-in component for the YOLO detection system
- Ad-YOLO introduces a patch class into the YOLO architecture to detect both objects of interest and adversarial patches
- Ad-YOLO outperforms YOLOv2 in white-box patch attacks, achieving 80.31% average precision (AP) for persons compared to 33.93% AP for YOLOv2
- Ad-YOLO has only a slight drop of 0.70% mean average precision (mAP) on the VOC 2007 test set compared to YOLOv2
- Ad-YOLO demonstrates excellent generalization ability compared to YOLOv2 in physical world attacks

## Summary

The security of object detection systems has become a major concern due to the rise of adversarial patch attacks. These attacks involve manipulating pixels in a specific area of an object, making them particularly effective against human detection systems. Existing defense mechanisms against patch attacks are primarily designed for image classification problems and struggle to defend against attacks on human detection systems. To address this issue, we propose a new defense strategy called Ad-YOLO, which is an efficient and effective plug-in component for the YOLO detection system. Ad-YOLO introduces a patch class into the YOLO architecture, allowing it to directly detect both objects of interest and adversarial patches with minimal impact on inference time. Our approach is the first defense strategy specifically targeted at human detection attacks. To evaluate the performance of Ad-YOLO, we conducted experiments using the YOLOv2 baseline. In order to improve Ad-YOLO's ability to detect various types of patches, we utilized an adversarial training process to create a patch dataset called Inria-Patch based on the Inria dataset. We then trained Ad-YOLO using a combination of Pascal VOC, Inria, and Inria-Patch datasets. The results demonstrate that Ad-YOLO outperforms YOLOv2 when facing white-box patch attacks, achieving an average precision (AP) of 80.31% for persons compared to 33.93% AP for YOLOv2 with only a slight drop of 0.70% mean average precision (mAP) on the VOC 2007 test set. Furthermore, we also evaluated Ad-YOLO's performance against physical world attacks to showcase its excellent generalization ability compared to YOLOv2. In conclusion, our proposed defense strategy - Ad-YOLO - effectively addresses the challenge posed by adversarial patch attacks in human detection systems by achieving superior performance compared to existing methods and demonstrating strong generalization capabilities.