# Bayesian Learning for Neural Networks: an algorithmic survey

**Authors:** Martin Magris, Alexandros Iosifidis  
**arXiv:** [2211.11865](https://arxiv.org/abs/2211.11865)  
**Date:** 2022-11-21  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2211.11865v1/

## Key points

- Comprehensive survey of Bayesian Learning for Neural Networks
- Focus on principles and algorithms involved in Bayesian learning
- Acknowledgement of limited widespread adoption due to technicality and complexity
- Aim to introduce topic from an accessible, practical-algorithmic perspective
- Challenges of computing posterior distribution in Bayesian paradigm
- Traditional sampling methods impractical for high number of parameters and large datasets
- Alternative estimation approaches like Variational Inference (VI) suitable for Bayesian inference
- Overview of methodologies available for Bayesian deep learning, with focus on VI methods
- Distinction between purely Bayesian methods and network architectures resembling a Bayesian framework
- Scope limited to purely Bayesian methods only
- Aims to fill a gap in existing literature by providing comprehensive overview of Bayesian learning methodologies for neural networks
- Hope to promote further research and applications in this area

## Summary

This preprint paper provides a comprehensive survey of Bayesian Learning for Neural Networks, focusing on the principles and algorithms involved in this approach. The authors acknowledge that while there has been a growing interest in Bayesian learning over the past decade, its technicality and complexity have limited its widespread adoption across different fields and applications. Therefore, this survey aims to introduce readers to the topic from an accessible, practical-algorithmic perspective. The paper begins by discussing the challenges of following the Bayesian paradigm, particularly in terms of computing the posterior distribution. Traditional sampling methods are often impractical due to the high number of parameters and large datasets typically encountered in machine learning. Instead, alternative estimation approaches such as Variational Inference (VI) have emerged as suitable and successful methods for Bayesian inference. The authors then delve into various methodologies available for Bayesian deep learning, presenting them from an algorithmic and empirically-oriented perspective. They specifically focus on VI methods but note that their goal is to provide a comprehensive overview of all major methodologies rather than solely focusing on VI. Additionally, the paper distinguishes between purely Bayesian methods and network architectures that resemble a Bayesian framework through techniques like creating output distributions or using ensembles; however, it limits its scope to purely Bayesian methods only. Overall, this survey aims to fill a gap in existing literature by providing a comprehensive overview of Bayesian learning methodologies for neural networks. By doing so, it hopes to promote further research and applications in this area which could potentially lead to wider adoption of these techniques across different fields and applications.