# Stochastic Activation Pruning for Robust Adversarial Defense

**Authors:** Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, Anima Anandkumar  
**arXiv:** [1803.01442](https://arxiv.org/abs/1803.01442)  
**Date:** 2018-03-05  
**Summary page:** https://www.summarizepaper.com/en/paper/1803.01442v1/

## Key points

- Neural networks vulnerable to adversarial examples
- Adversarial examples undermine reliability of deep learning systems
- Problem framed as minimax zero-sum game between adversary and model
- Proposed solution: Stochastic Activation Pruning (SAP)
- SAP involves randomly pruning subset of activations with preference for smaller magnitudes, scaling up remaining activations
- SAP can be applied to pretrained networks without fine-tuning
- Experimental results show SAP improves robustness against attacks, enhances accuracy, preserves calibration
- SAP provides effective defense mechanism against adversarial examples in deep learning systems

## Summary

Neural networks have been shown to be vulnerable to adversarial examples, where carefully crafted perturbations of real images can lead to misclassification and undermine the reliability of deep learning systems. To address this issue, the authors draw inspiration from game theory and frame the problem as a minimax zero-sum game between the adversary and the model. They propose a mixed strategy called Stochastic Activation Pruning (SAP) for robust adversarial defense. SAP involves randomly pruning a subset of activations with preference for those with smaller magnitudes while scaling up the remaining activations to compensate. The advantage of SAP is that it can be applied to pretrained networks including adversarially trained models without requiring fine-tuning. Experimental results demonstrate that SAP improves robustness against attacks by enhancing accuracy and preserving calibration. This approach provides an effective defense mechanism against adversarial examples in deep learning systems.