# Benchmarking the Physical-world Adversarial Robustness of Vehicle Detection

**Authors:** Tianyuan Zhang, Yisong Xiao, Xiaoya Zhang, Hao Li, Lu Wang  
**arXiv:** [2304.05098](https://arxiv.org/abs/2304.05098)  
**Date:** 2023-04-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.05098v1/

## Key points


## Summary

The paper titled "Benchmarking the Physical-world Adversarial Robustness of Vehicle Detection" by Tianyuan Zhang, Yisong Xiao, Xiaoya Zhang, Hao Li, and Lu Wang discusses the challenges associated with evaluating the robustness of detection models in the physical world due to its time-consuming and labor-intensive nature. To address this issue, virtual simulation experiments can provide a solution. However, there is no unified detection benchmark based on virtual simulation environment. To overcome this challenge, the authors proposed an instant-level data generation pipeline based on the CARLA simulator which resulted in the DCI dataset. This dataset covers 7 continuous and 1 discrete scenes with over 40 angles, 20 distances and 20,000 positions. Extensive experiments were conducted on three detection models and three physical adversarial attacks using this dataset. The results indicate that Yolo v6 had strongest resistance with only a 6.59% average AP drop while ASA was found to be the most effective attack algorithm with a 14.51% average AP reduction - twice that of other algorithms tested. Static scenes had higher recognition AP while results under different weather conditions were similar. Moreover, it was observed that adversarial attack algorithm improvement may be approaching its 'limitation'. Overall, this study provides valuable insights into evaluating the robustness of detection models in virtual simulation environments using instant-level data generation pipelines like CARLA simulator-based DCI dataset which could serve as a unified benchmark for future research in this area.