# Invariant Information Clustering for Unsupervised Image Classification and Segmentation

**Authors:** Xu Ji, João F. Henriques, Andrea Vedaldi  
**arXiv:** [1807.06653](https://arxiv.org/abs/1807.06653)  
**Date:** 2018-07-17  
**Summary page:** https://www.summarizepaper.com/en/paper/1807.06653v4/

## Key points

- The paper presents a novel clustering objective for unsupervised image classification and segmentation using unlabelled data samples.
- The objective is to discover clusters that accurately match semantic classes, achieving state-of-the-art results in unsupervised clustering benchmarks.
- The proposed method outperforms competitors by a significant margin in benchmarks such as STL10 and CIFAR10.
- The approach is versatile and can be applied to any paired dataset samples, not limited to computer vision tasks.
- Random transforms are used to obtain pairs from each image in experiments.
- Unlike other methods, the trained network directly outputs semantic labels without requiring external processing for semantic clustering.
- The main objective is to maximize mutual information between class assignments of each pair, avoiding degenerate solutions.
- Two semi-supervised settings are explored, achieving impressive accuracy rates on STL10 classification and robustness to label coverage reductions up to 90%.
- Overall, the paper introduces a powerful clustering objective that learns a neural network classifier without relying on labeled data.

## Summary

The paper titled "Invariant Information Clustering for Unsupervised Image Classification and Segmentation" presents a novel clustering objective that aims to learn a neural network classifier from scratch using only unlabelled data samples. The objective of this model is to discover clusters that accurately match semantic classes, achieving state-of-the-art results in various unsupervised clustering benchmarks related to image classification and segmentation. The authors demonstrate the effectiveness of their approach by evaluating it on eight different benchmarks, including STL10 (an unsupervised variant of ImageNet) and CIFAR10. In both cases, the proposed method outperforms its closest competitors by a significant margin, achieving improvements of 6.6 and 9.5 absolute percentage points respectively. One notable aspect of this approach is its versatility, as it is not specialized solely for computer vision tasks but can operate on any paired dataset samples. To obtain pairs from each image in their experiments, the authors employ random transforms. Moreover, unlike other methods that produce high-dimensional representations requiring external processing for semantic clustering, the trained network directly outputs semantic labels. The main objective of the proposed method is to maximize mutual information between the class assignments of each pair. This objective is grounded in information theory and offers several advantages over other clustering methods by avoiding degenerate solutions. In addition to the fully unsupervised mode, the authors also explore two semi-supervised settings. In one setting, they achieve an impressive accuracy rate of 88.8% on STL10 classification, setting a new global state-of-the-art across all existing methods (supervised, semi-supervised or unsupervised). In another setting, they demonstrate robustness to label coverage reductions up to 90%, which is particularly relevant for applications that rely on small amounts of labeled data. Overall, this paper introduces a powerful clustering objective that effectively learns a neural network classifier without relying on labeled data. The experimental results showcase its superiority over existing methods in various unsupervised clustering benchmarks, making it a valuable contribution to the field of image classification and segmentation.