# No More Pesky Learning Rates

**Authors:** Tom Schaul, Sixin Zhang, Yann LeCun  
**arXiv:** [1206.1106](https://arxiv.org/abs/1206.1106)  
**Date:** 2012-06-06  
**Summary page:** https://www.summarizepaper.com/fr/paper/1206.1106v1/

## Abstract

The performance of stochastic gradient descent (SGD) depends critically on how learning rates are tuned and decreased over time. We propose a method to automatically adjust multiple learning rates so as to minimize the expected error at any one time. The method relies on local gradient variations across samples. Using a number of convex and non-convex learning tasks, we show that the resulting algorithm matches the performance of the best settings obtained through systematic search, and effectively removes the need for learning rate tuning.