Search Results for « Bayesian Deep Learning »

We showcase how dropout variational inference can be applied to a large-scale deep learning model that predicts price movements from limit order books (LOBs), the canonical data source representing trading and pricing movements. We demonstrate that uncertainty information derived from posterior predictive distributions can be utilised for position sizing, avoiding unnecessary trades and improving profits. Further, we test our models by using millions of observations across several instruments and markets from the London Stock Exchange. Our results suggest that those Bayesian techniques not only deliver uncertainty information that can be used for trading but also improve predictive performance as stochastic regularisers. To the best of our knowledge, we are the first to apply Bayesian networks to LOBs.

Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals. Given the many aspects of an experiment it is always possible that minor or even major experimental flaws can slip by both authors and reviewers. One of the most popular experiments used to evaluate approximate inference techniques is the regression experiment on UCI datasets. However, in this experiment, models which have been trained to convergence have often been compared with baselines trained only for a fixed number of iterations. We find that a well-established baseline, Monte Carlo dropout, when evaluated under the same experimental settings shows significant improvements. In fact, the baseline outperforms or performs competitively with methods that claimed to be superior to the very same baseline method when they were introduced. Hence, by exposing this flaw in experimental procedure, we highlight the importance of using identical experimental setups to evaluate, compare, and benchmark methods in Bayesian Deep Learning.

Abstract Traffic flow distribution forecasting plays a crucial role in intelligent transportation systems for effective traffic management and congestion mitigation. In this paper, we propose a novel approach for traffic flow distribution forecasting using federated learning and Bayesian deep learning. The proposed method leverages the collaborative training capability of federated learning to learn from decentralized data sources while preserving data privacy and ownership. Bayesian deep learning techniques are integrated to estimate uncertainties and provide probabilistic predictions, enabling the capture of the inherent variability in traffic flow distributions. We conduct comprehensive experiments on real-world traffic flow datasets and compare our method with baseline approaches. The results demonstrate that our proposed method outperforms the baselines in terms of mean absolute error, root mean squared error, negative log-likelihood, and calibration error. The integration of federated learning and Bayesian deep learning enables accurate traffic flow distribution forecasts while providing reliable uncertainty estimates. This work contributes to the advancement of intelligent transportation systems and provides valuable insights for decision-making in traffic management and congestion reduction.

Evidential deep learning, built upon belief theory and subjective logic, offers a principled and computationally efficient way to turn a deterministic neural network uncertainty-aware. The resultant evidential models can quantify fine-grained uncertainty using the learned evidence. To ensure theoretically sound evidential models, the evidence needs to be non-negative, which requires special activation functions for model training and inference. This constraint often leads to inferior predictive performance compared to standard softmax models, making it challenging to extend them to many large-scale datasets. To unveil the real cause of this undesired behavior, we theoretically investigate evidential models and identify a fundamental limitation that explains the inferior performance: existing evidential activation functions create zero evidence regions, which prevent the model to learn from training samples falling into such regions. A deeper analysis of evidential activation functions based on our theoretical underpinning inspires the design of a novel regularizer that effectively alleviates this fundamental limitation. Extensive experiments over many challenging real-world datasets and settings confirm our theoretical findings and demonstrate the effectiveness of our proposed approach.

We describe the new field of mathematical analysis of deep learning. This field emerged around a list of research questions that were not answered within the classical framework of learning theory. These questions concern: the outstanding generalization power of overparametrized neural networks, the role of depth in deep architectures, the apparent absence of the curse of dimensionality, the surprisingly successful optimization performance despite the non-convexity of the problem, understanding what features are learned, why deep architectures perform exceptionally well in physical problems, and which fine aspects of an architecture affect the behavior of a learning task in which way. We present an overview of modern approaches that yield partial answers to these questions. For selected approaches, we describe the main ideas in more detail.

One of the main challenges of deep learning tools is their inability to capture model uncertainty. While Bayesian deep learning can be used to tackle the problem, Bayesian neural networks often require more time and computational power to train than deterministic networks. Our work explores whether fully Bayesian networks are needed to successfully capture model uncertainty. We vary the number and position of Bayesian layers in a network and compare their performance on active learning with the MNIST dataset. We found that we can fully capture the model uncertainty by using only a few Bayesian layers near the output of the network, combining the advantages of deterministic and Bayesian networks.

<p><b>Inverse problems are among the most challenging and widespread problems in science today. Inverse problems are commonly ill-conditioned, having potentially infinitely many solutions, and prior assumptions must be made in order to make the solution unique. Techniques for solving inverse problems are typically bespoke, requiring different tools and frameworks for each problem.</b></p> <p>In this thesis we use Bayes' rule as a generic solution to a range of inverse problems, coupled with the deep learning technique known as normalising flows to provide models for the prior. Bayes' provides great flexibility -- being easily applicable to a range of inverse problems -- as well as results competitive with, or even exceeding, state-of-the-art techniques in image reconstruction, MRI acceleration and radio astronomy.</p>