# Minimum Relative Entropy Inference for Normal and Monte Carlo Distributions

**Authors:** Marcello Colasante, Attilio Meucci  
**arXiv:** [2007.06461](https://arxiv.org/abs/2007.06461)  
**Date:** 2020-07-13  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2007.06461v1/

## Key points

- Investigating representation of affine sub-manifolds of exponential family distributions as minimum relative entropy sub-manifolds
- Deriving analytical formulas for inference from partial information on expectations and covariances of multivariate normal distributions
- Enhancing numerical implementation through Monte Carlo simulations for inference from partial information of generalized expectation type
- Introducing target vector X and random vector Z to represent base distribution and new information input variables
- Utilizing minimum relative entropy (MRE) as a standard method for inference with partial information
- Applying and comparing analytical solutions and numerical algorithms in a case study to showcase effectiveness in handling inference tasks based on partial information
- Solving the MRE problem under exponential-family base distributions and expectation type constraints using analytical solutions under normal base distributions and numerical computations under more general base distributions via iterative Hamiltonian Monte Carlo simulations
- Providing improved approximations of updated distributions compared to previous algorithms by utilizing refined approaches based on minimum relative entropy principles

## Summary

We investigate the representation of affine sub-manifolds of exponential family distributions as minimum relative entropy sub-manifolds. This allows us to derive analytical formulas for inference from partial information on expectations and covariances of multivariate normal distributions. We also enhance the numerical implementation through Monte Carlo simulations for the inference from partial information of generalized expectation type. To provide context, we introduce the target vector X and random vector Z, representing the base distribution and new information input variables respectively. Our approach utilizes minimum relative entropy (MRE) as a standard method for inference with partial information. The principle of MRE guides us in updating inferred distributions that are closest to the base distribution while satisfying constraints imposed by the inference variables. In our case study (Section 5), we apply and compare both analytical solutions and numerical algorithms, showcasing their effectiveness in handling inference tasks based on partial information. Additionally, we summarize our main contributions (Section 6) which include solving the MRE problem under exponential-family base distributions and expectation type constraints using analytical solutions under normal base distributions and numerical computations under more general base distributions via iterative Hamiltonian Monte Carlo simulations. These methods provide improved approximations of updated distributions compared to previous algorithms. Overall, this research highlights the significance of utilizing minimum relative entropy principles in statistical inference tasks involving partial information constraints. Our refined approaches offer valuable insights into efficiently updating distributions based on limited data inputs, paving the way for enhanced decision-making processes in various applications.