# Epistemic Neural Networks

**Authors:** Ian Osband, Zheng Wen, Mohammad Asghari, Morteza Ibrahimi, Xiyuan Lu, Benjamin Van Roy  
**arXiv:** [2107.08924](https://arxiv.org/abs/2107.08924)  
**Date:** 2021-07-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2107.08924v1/

## Key points

- Authors introduce Epistemic Neural Networks (ENN) as a novel interface for uncertainty modeling in deep learning
- ENN framework encompasses all existing approaches to uncertainty modeling, equating each ENN to a Bayesian neural network
- Shifts focus from developing probabilistic inference tools for neural networks to identifying best-suited neural networks for probabilistic inference
- Propose KL-divergence as a metric for assessing progress in ENNs
- Develop computational testbed for evaluating performance of various uncertainty modeling approaches in deep learning
- Significant variations in performance observed among different approaches
- High correlation between proposed metric and performance in sequential decision-making tasks demonstrated
- New ENN architectures have potential to enhance statistical quality and computational efficiency
- Study emphasizes importance of ENNs as tools for uncertainty modeling in deep learning, paving the way for advancements in leveraging neural networks for probabilistic inference tasks

## Summary

Authors Ian Osband, Zheng Wen, Mohammad Asghari, Morteza Ibrahimi, Xiyuan Lu, and Benjamin Van Roy introduce the concept of Epistemic Neural Networks (ENN) as a novel interface for uncertainty modeling in deep learning. They highlight that all existing approaches to uncertainty modeling can be framed within the ENN framework, with each ENN being equated to a Bayesian neural network. This fresh perspective opens up promising avenues for future research by shifting the focus from developing probabilistic inference tools for neural networks to identifying which neural networks are best suited as tools for probabilistic inference. The authors propose a straightforward and effective metric for assessing progress in ENNs: the KL-divergence with respect to a target distribution. To evaluate the performance of various canonical approaches to uncertainty modeling in deep learning, they develop a computational testbed centered on inference in a neural network Gaussian process. The code for this testbed is made publicly available as a benchmark on GitHub. Through their evaluation, the authors discover significant variations in performance among different uncertainty modeling approaches. They delve into the sensitivity of these results and demonstrate a high correlation between their proposed metric and performance in sequential decision-making tasks. Additionally, they provide insights suggesting that new ENN architectures have the potential to enhance both statistical quality and computational efficiency. In conclusion, this study by Osband et al. sheds light on the importance of Epistemic Neural Networks as tools for uncertainty modeling in deep learning. Their findings not only contribute valuable insights into current methodologies but also pave the way for further advancements in leveraging neural networks for probabilistic inference tasks.