# Architectural Backdoors in Neural Networks

**Authors:** Mikel Bober-Irizar, Ilia Shumailov, Yiren Zhao, Robert Mullins, Nicolas Papernot  
**arXiv:** [2206.07840](https://arxiv.org/abs/2206.07840)  
**Date:** 2022-06-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.07840v1/

## Key points

- Machine learning models are vulnerable to adversarial manipulation
- Attackers can manipulate data and data sampling procedures during the training stage to control model behavior
- Common attack goal is to plant backdoors that force the victim model to recognize a trigger known only by the adversary
- A new class of backdoor attacks hide inside model architectures, specifically in the inductive bias of the functions used to train
- Architectural backdoors can survive complete re-training from scratch, making them particularly insidious and difficult to detect
- The paper formalizes the main construction principles behind architectural backdoors, such as a link between input and output, and describes some possible protections against them.
- The authors evaluate their attacks on computer vision benchmarks of different scales and demonstrate that this underlying vulnerability is pervasive across various training settings.

## Summary

The paper "Architectural Backdoors in Neural Networks" explores the vulnerability of machine learning models to adversarial manipulation. The authors highlight that attackers can manipulate data and data sampling procedures during the training stage to control model behavior, with a common attack goal being to plant backdoors that force the victim model to recognize a trigger known only by the adversary. The authors introduce a new class of backdoor attacks that hide inside model architectures, specifically in the inductive bias of the functions used to train. These backdoors are simple to implement, such as by publishing open-source code for a backdoored model architecture that others will reuse unknowingly. Unlike other approaches, architectural backdoors can survive complete re-training from scratch, making them particularly insidious and difficult to detect. The paper formalizes the main construction principles behind architectural backdoors, such as a link between input and output, and describes some possible protections against them. The authors evaluate their attacks on computer vision benchmarks of different scales and demonstrate that this underlying vulnerability is pervasive across various training settings. Overall, this research highlights an important aspect of machine learning security: not only must we be vigilant about data manipulation but also about potential vulnerabilities hidden within model architectures themselves. By raising awareness of these issues and proposing possible solutions, this work contributes significantly towards building more robust and secure machine learning systems.