# Towards Stable Test-Time Adaptation in Dynamic Wild World

**Authors:** Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, Mingkui Tan  
**arXiv:** [2302.12400](https://arxiv.org/abs/2302.12400)  
**Date:** 2023-02-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.12400v1/

## Key points

- The paper investigates the effectiveness of test-time adaptation (TTA) in addressing distribution shifts between training and testing data.
- TTA's online model updating can be unstable, hindering its deployment in real-world scenarios.
- Batch norm layer is a crucial factor contributing to this instability.
- TTA can perform more stably with batch-agnostic norm layers such as group or layer norm, but still suffers many failure cases.
- Noisy test samples with large gradients may disturb the model adaption and result in collapsed trivial solutions where all samples are assigned the same class label.
- A sharpness-aware and reliable entropy minimization method called SAR stabilizes TTA from two aspects: removing partial noisy samples with large gradients and encouraging model weights to go to a flat minimum so that it is robust to remaining noisy samples.
- The proposed method demonstrates better performance than prior methods and is computationally efficient under wild test scenarios.

## Summary

The paper titled "Towards Stable Test-Time Adaptation in Dynamic Wild World" by Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao and Mingkui Tan investigates the effectiveness of test-time adaptation (TTA) in addressing distribution shifts between training and testing data. While TTA has shown promising results in adapting a given model on test samples, its online model updating can be unstable which hinders its deployment in real-world scenarios. The authors identify that the batch norm layer is a crucial factor contributing to this instability and propose that TTA can perform more stably with batch-agnostic norm layers such as group or layer norm. However, they observe that even with these norms TTA still suffers many failure cases. By investigating these failure cases further, the authors find that certain noisy test samples with large gradients may disturb the model adaption and result in collapsed trivial solutions where all samples are assigned the same class label. To address this collapse issue they propose a sharpness-aware and reliable entropy minimization method called SAR which stabilizes TTA from two aspects: 1) removing partial noisy samples with large gradients and 2) encouraging model weights to go to a flat minimum so that it is robust to remaining noisy samples. The proposed method demonstrates better performance than prior methods and is computationally efficient under wild test scenarios where mixed distribution shifts, small batch sizes, and online imbalanced label distribution shifts are common. This paper provides valuable insights into unstable reasons behind TTA's performance issues while also presenting an effective solution for stabilizing it under challenging real-world conditions.