# Gradient Estimation with Discrete Stein Operators

**Authors:** Jiaxin Shi, Yuhao Zhou, Jessica Hwang, Michalis K. Titsias, Lester Mackey  
**arXiv:** [2202.09497](https://arxiv.org/abs/2202.09497)  
**Date:** 2022-02-19  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2202.09497v1/

## Key points

- Gradient estimation is crucial in machine learning for approximating the gradient of an expectation with respect to distribution parameters.
- Common gradient estimators suffer from excessive variance when dealing with discrete distributions.
- The authors propose a variance reduction technique using Stein operators for discrete distributions.
- The technique involves constructing control variates for the REINFORCE leave-one-out estimator using Stein operators.
- These control variates can be adapted online to minimize variance without additional evaluations of the target function.
- The proposed gradient estimator achieves significantly lower variance compared to state-of-the-art estimators in benchmark generative modeling tasks.
- The approach improves accuracy and efficiency of gradient estimation for discrete distributions.

## Summary

In the field of machine learning, gradient estimation plays a crucial role in approximating the gradient of an expectation with respect to the parameters of a distribution. However, when dealing with discrete distributions, common gradient estimators often suffer from excessive variance. To address this issue and enhance the quality of gradient estimation, the authors propose a variance reduction technique based on Stein operators specifically designed for discrete distributions. The technique introduced by the authors involves utilizing Stein operators to construct flexible control variates for the REINFORCE leave-one-out estimator. These control variates can be adapted online to minimize variance without requiring additional evaluations of the target function. By incorporating these control variates into benchmark generative modeling tasks such as training binary variational autoencoders, the proposed gradient estimator achieves significantly lower variance compared to state-of-the-art estimators while using the same number of function evaluations. Overall, this research presents a novel approach to improving gradient estimation in machine learning problems involving discrete distributions. The use of Stein operators and adaptive control variates offers an effective means of reducing variance and enhancing the accuracy and efficiency of gradient estimation techniques for discrete distributions.