# Soft Augmentation for Image Classification

**Authors:** Yang Liu, Shen Yan, Laura Leal-Taixé, James Hays, Deva Ramanan  
**arXiv:** [2211.04625](https://arxiv.org/abs/2211.04625)  
**Date:** 2022-11-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.04625v1/

## Key points

- Neural networks in deep learning have become complex and over-parameterized.
- Strong regularization techniques like data augmentation and weight decay are used to address overfitting and improve generalization.
- Data augmentation involves applying transformations to training samples to create additional variations of the data.
- Soft augmentation is a new approach that introduces a non-linear softening effect on the learning target based on the degree of transformation applied to each sample.
- Soft targets allow for more aggressive data augmentation strategies and offer more robust performance improvements compared to traditional approaches.
- Soft targets also improve model calibration by training models to be less confident on aggressively cropped or occluded examples.
- Soft target-based methods demonstrate significant improvements across various benchmark datasets, including doubling top-1 accuracy boost, up to four times better model occlusion performance, and halving expected calibration error (ECE).
- Soft augmentation techniques can be generalized to self-supervised classification tasks beyond traditional supervised settings.

## Summary

In the field of deep learning, neural networks have become increasingly complex and over-parameterized. To address the issue of overfitting and improve generalization, strong regularization techniques such as data augmentation and weight decay are commonly employed. Data augmentation involves applying various transformations to training samples to create additional variations of the data. Traditionally, data augmentation has focused on invariant transforms, where the learning target remains unchanged regardless of the applied transformation. However, drawing inspiration from human visual classification studies, a new approach called soft augmentation is proposed. Soft augmentation introduces a non-linear softening effect on the learning target based on the degree of transformation applied to each sample. For example, more aggressive image crop augmentations result in less confident learning targets. The use of soft targets allows for more aggressive data augmentation strategies and offers more robust performance improvements compared to traditional approaches. Soft targets also work well with other augmentation policies and have an interesting side effect – they produce better calibrated models. By training models to be less confident on aggressively cropped or occluded examples, soft targets improve model calibration. When combined with existing aggressive augmentation strategies, soft target-based methods demonstrate significant improvements across various benchmark datasets such as Cifar-10, Cifar-100, ImageNet-1K, and ImageNet-V2. These improvements include doubling the top-1 accuracy boost compared to traditional methods, up to four times better model occlusion performance, and halving the expected calibration error (ECE). Furthermore, it is shown that soft augmentation techniques can be generalized to self-supervised classification tasks beyond traditional supervised settings. Overall, this research highlights the effectiveness of soft augmentation in addressing overfitting issues in modern neural networks. By introducing a non-linear relationship between transformations and learning targets, soft targets enable more aggressive data augmentation while improving model performance and calibration.