Unbiased Derivative Estimation for Stationary Mean of Parameterized Markov chains

2026-06-09

AI Summary

This work addresses the inefficiency of existing methods for estimating gradients of stationary means in slowly mixing parametrized Markov chains. The authors propose a general-purpose, unbiased gradient estimator that requires only black-box access to the transition kernel density and its gradient, without assuming any specific parametric form—making it applicable to complex models such as neural networks. By integrating stochastic differentiation with control variate techniques, the method achieves, for the first time, efficient and unbiased estimation of gradients of stationary means for any differentiable parametrization of Markov chains. Theoretical analysis demonstrates that the proposed estimator substantially outperforms current approaches in slow-mixing regimes, and numerical experiments corroborate its superior empirical performance.

Abstract

We propose a new approach to unbiased estimation of the gradients of the stationary means associated with parametrized families of Markov chains. Our estimators are particularly efficient when the Markov chains have slow mixing rate. Our approach does not require a specific parametrization except for an oracle to evaluate the transition density and its gradient at a given data point without any additional knowledge about the density function itself. It makes our estimator suitable for parametrizations associated with neural networks. The estimator can potentially achieve large improvement in terms of efficiency. Numerical experiments confirm the good performance predicted by the theory.

Problem

• unbiased derivative estimation
• stationary mean
• parameterized Markov chains
• slow mixing
• gradient estimation

Innovation

• unbiased gradient estimation
• stationary mean
• parameterized Markov chains
• slow mixing
• neural network parametrization