# Anomaly Detection via Reverse Distillation from One-Class Embedding

**Authors:** Hanqiu Deng, Xingyu Li  
**arXiv:** [2201.10703](https://arxiv.org/abs/2201.10703)  
**Date:** 2022-01-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.10703v2/

## Key points

- Knowledge distillation (KD) is effective for unsupervised anomaly detection (AD)
- Anomalies show representation discrepancy in teacher-student (T-S) model
- Novel T-S model proposed with teacher encoder and student decoder
- "Reverse distillation" paradigm introduced where student network takes one-class embedding from teacher model as input
- Student reconstructs teacher's multiscale representations from high-level to low-level features
- Trainable one-class bottleneck embedding (OCBE) module integrated into T-S model
- Extensive experimentation shows approach surpasses state-of-the-art performance levels in AD and one-class novelty detection benchmarks

## Summary

Knowledge distillation (KD) has shown promising results in addressing the challenging task of unsupervised anomaly detection (AD). Anomalies often exhibit a representation discrepancy in the teacher-student (T-S) model, which serves as crucial evidence for AD. However, previous studies have encountered limitations due to the use of similar or identical architectures for both the teacher and student models, leading to a lack of diversity in anomalous representations. To overcome this issue, a novel T-S model is proposed in this study, comprising a teacher encoder and a student decoder. A unique "reverse distillation" paradigm is introduced where the student network takes the one-class embedding from the teacher model as input instead of directly receiving raw images. The goal is for the student to reconstruct the teacher's multiscale representations starting from abstract high-level presentations to low-level features. Furthermore, a trainable one-class bottleneck embedding (OCBE) module is integrated into the T-S model. This compact embedding effectively retains essential information on normal patterns while filtering out anomaly perturbations. Extensive experimentation on AD and one-class novelty detection benchmarks demonstrates that this approach surpasses state-of-the-art performance levels. The study by Hanqiu Deng and Xingyu Li presents an innovative methodology for anomaly detection through reverse distillation from one-class embedding. Published in CVPR 2022 with 10 pages and 7 figures, their research showcases the effectiveness and generalizability of their proposed approach in enhancing anomaly detection capabilities.