# Learning deep representations by mutual information estimation and maximization

**Authors:** R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio  
**arXiv:** [1808.06670](https://arxiv.org/abs/1808.06670)  
**Date:** 2018-08-20  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1808.06670v5/

## Key points

- Authors: R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio
- Approach: Maximizing mutual information between input and output in a deep neural network encoder
- Importance of structure in representation learning
- Incorporating knowledge about input locality enhances effectiveness of representations for downstream tasks
- Introduction of Deep InfoMax (DIM) method that outperforms popular unsupervised learning techniques and competes with fully-supervised learning on multiple classification tasks
- Use of adversarial matching to control key characteristics of representations according to a prior distribution
- Impact: Accepted as an oral presentation at the International Conference for Learning Representations (ICLR) in 2019

## Summary

In their paper titled "Learning deep representations by mutual information estimation and maximization," authors R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio explore the realm of unsupervised learning of representations through a novel approach of maximizing mutual information between input and output in a deep neural network encoder. The study highlights the importance of structure in representation learning and showcases how incorporating knowledge about input locality can greatly enhance the effectiveness of representations for downstream tasks. Additionally, the authors introduce Deep InfoMax (DIM), a method that not only outperforms popular unsupervised learning techniques but also competes with fully-supervised learning on multiple classification tasks. Furthermore, they push the boundaries by using adversarial matching to control key characteristics of representations according to a prior distribution. This groundbreaking methodology opens up new possibilities for unsupervised representation learning and marks a significant step towards formulating flexible objectives tailored to specific end-goals in representation-learning tasks. The paper was accepted as an oral presentation at the International Conference for Learning Representations (ICLR) in 2019, highlighting its relevance and impact within the machine learning community.