# Provable convergence guarantees for black-box variational inference

**Authors:** Justin Domke, Guillaume Garrigos, Robert Gower  
**arXiv:** [2306.03638](https://arxiv.org/abs/2306.03638)  
**Date:** 2023-06-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.03638v1/

## Key points

- Lack of proof regarding the success of stochastic optimization in black-box variational inference
- Challenges posed by gradient estimators with unusual noise bounds and a composite non-smooth objective
- Focus on dense Gaussian variational families and existing gradient estimators based on reparameterization satisfying a quadratic noise bound
- Novel convergence guarantees for proximal and projected stochastic gradient descent using this bound
- First rigorous guarantee that black-box variational inference can converge for realistic inference problems
- Bridging a theoretical gap in existing stochastic optimization proofs
- Implications for the field of machine learning, as black-box variational inference is widely used but lacked formal proof of its effectiveness
- Authored by Justin Domke, Guillaume Garrigos, and Robert Gower
- 32 pages long and falls under the categories of cs.LG, math.OC, and stat.ML

## Summary

The paper titled "Provable convergence guarantees for black-box variational inference" addresses the lack of proof regarding the success of stochastic optimization in black-box variational inference. The authors argue that this gap exists due to challenges posed by gradient estimators with unusual noise bounds and a composite non-smooth objective. To address this issue, the authors focus on dense Gaussian variational families and observe that existing gradient estimators based on reparameterization satisfy a quadratic noise bound. They further provide novel convergence guarantees for proximal and projected stochastic gradient descent using this bound. This research breakthrough is significant as it offers the first rigorous guarantee that black-box variational inference can converge for realistic inference problems. By establishing these provable convergence guarantees, the authors bridge a theoretical gap in existing stochastic optimization proofs. This finding has important implications for the field of machine learning, as black-box variational inference is widely used but has lacked formal proof of its effectiveness. The paper is authored by Justin Domke, Guillaume Garrigos, and Robert Gower. It spans 32 pages and falls under the categories of cs.LG (Computer Science - Machine Learning), math.OC (Mathematics - Optimization and Control), and stat.ML (Statistics - Machine Learning).