# Extremely Simple Activation Shaping for Out-of-Distribution Detection

**Authors:** Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, Rosanne Liu  
**arXiv:** [2209.09858](https://arxiv.org/abs/2209.09858)  
**Date:** 2022-09-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2209.09858v1/

## Key points

- Introduction of ASH, a novel activation shaping method for enhancing in-distribution (ID) and out-of-distribution (OOD) sample distinction
- ASH improves OOD detection performance on ImageNet by modifying input sample activations at inference time
- Two calls for explanation and validation are issued to explore the effectiveness and applicability of ASH:
- Call for explanation: Investigating reasons why ASH works well, suggesting neural networks' overparameterization may lead to redundant features hindering discrimination between seen and unseen data
- Call for validation: Encouraging research in other domains like natural language processing with transformer-based models
- ASH demonstrates superior performance in OOD detection while maintaining high ID classification accuracy through experiments on multiple datasets
- The unexpected success of ASH prompts further investigation into its mechanisms and potential applications across various research domains

## Summary

In this paper, the authors introduce ASH, a novel activation shaping method that enhances in-distribution (ID) and out-of-distribution (OOD) sample distinction without significantly impacting in-distribution accuracy. The motivation behind ASH stems from the limitations of solely relying on advancements in training to anticipate all scenarios encountered during deployment of machine learning models. By removing a large portion of an input sample's activation at a late layer and simplifying or lightly adjusting the remaining portion at inference time, ASH effectively improves OOD detection performance on ImageNet. The authors also issue two calls for explanation and validation to further explore the effectiveness and applicability of ASH. The call for explanation seeks plausible reasons for why ASH works well, suggesting that overparameterized neural networks may generate redundant features that hinder discrimination between seen and unseen data. On the other hand, the call for validation encourages researchers to investigate other domains where similar techniques could be applied, such as natural language processing with transformer-based language models. Through extensive experiments on multiple ID and OOD datasets, ASH demonstrates superior performance compared to contemporary methods for OOD detection while maintaining high ID classification accuracy. The unexpected success of ASH prompts further investigation into its underlying mechanisms, prompting collaboration with fellow researchers to delve deeper into its potential applications and implications across various research domains. Overall, ASH presents a promising approach to improving model robustness in handling unforeseen scenarios during deployment.