# Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

**Authors:** Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko  
**arXiv:** [2006.07733](https://arxiv.org/abs/2006.07733)  
**Date:** 2020-06-13  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2006.07733v1/

## Key points

- "Bootstrap Your Own Latent" (BYOL) is a new approach to self-supervised image representation learning
- BYOL uses two neural networks, an online and a target network, that interact and learn from each other
- The online network is trained to predict the target network representation of an augmented view of an image
- The target network is updated with a slow-moving average of the online network
- BYOL achieves 74.3% top-1 classification accuracy on ImageNet using the standard linear evaluation protocol with a ResNet-50 architecture and 79.6% with a larger ResNet
- BYOL performs on par or better than current state-of-the-art methods on both transfer and semi-supervised benchmarks
- BYOL's success may be due in part to its ability to leverage large amounts of unlabeled data for pre-training tasks such as contrastive learning without requiring negative pairs
- BYOL's reliance on only positive pairs may make it more robust to dataset biases

## Summary

In their paper "Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning," Jean-Bastien Grill and colleagues introduce a novel approach to self-supervised image representation learning called Bootstrap Your Own Latent (BYOL). The method relies on two neural networks, an online and a target network, that interact and learn from each other. Using an augmented view of an image, the online network is trained to predict the target network representation of the same image under a different augmented view. At the same time, the target network is updated with a slow-moving average of the online network. While state-of-the-art methods rely on negative pairs, BYOL achieves a new state of the art without them. In fact, BYOL reaches 74.3% top-1 classification accuracy on ImageNet using the standard linear evaluation protocol with a ResNet-50 architecture and 79.6% with a larger ResNet. The authors also show that BYOL performs on par or better than current state-of-the-art methods on both transfer and semi-supervised benchmarks. Furthermore, they note that BYOL's success may be due in part to its ability to leverage large amounts of unlabeled data for pre-training tasks such as contrastive learning without requiring negative pairs. Additionally, BYOL's reliance on only positive pairs may make it more robust to dataset biases. Overall, this new approach offers promising results for self-supervised learning in computer vision tasks and could have implications for improving performance in downstream applications such as object recognition and detection.