# Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?

**Authors:** Kyurae Kim, Yian Ma, Jacob R. Gardner  
**arXiv:** [2307.14642](https://arxiv.org/abs/2307.14642)  
**Date:** 2023-07-27  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2307.14642v1/

## Key points

- Black-box variational inference (BBVI) with control variates achieves geometric convergence under perfect variational family specification
- Quadratic bound on the gradient variance of the sticking-the-landing (STL) estimator extends to misspecified variational families
- Convergence of BBVI using projected stochastic gradient descent is implied by the quadratic variance condition
- Analysis of regular closed-form entropy gradient estimators is improved upon and compared to the STL estimator
- Explicit non-asymptotic complexity guarantees are provided for both methods
- Contributes to understanding of BBVI and its convergence properties, offering insights into gradient variance bounds
- Provides valuable guidance for practitioners working with variational inference methods

## Summary

This paper presents a proof that black-box variational inference (BBVI) with control variates, specifically the sticking-the-landing (STL) estimator, achieves geometric convergence under perfect variational family specification. The authors establish a quadratic bound on the gradient variance of the STL estimator which extends to encompass misspecified variational families. This result combined with previous research on the quadratic variance condition implies the convergence of BBVI when using projected stochastic gradient descent. Additionally, this study improves upon existing analysis of regular closed-form entropy gradient estimators and enables a comparison between these estimators and the STL estimator while providing explicit non-asymptotic complexity guarantees for both methods. Overall, this paper contributes to our understanding of BBVI and its convergence properties by demonstrating geometric convergence under certain conditions and offering insights into gradient variance bounds. These theoretical foundations provide valuable guidance for practitioners working with variational inference methods.