Gradient estimators for parameter inference in discrete stochastic kinetic models

2026-04-02

AI Summary

This study addresses the challenge of parameter inference in discrete stochastic dynamical models—such as those simulated via the Gillespie algorithm—where non-differentiability impedes gradient-based learning. The authors systematically introduce and compare three gradient estimators: Gumbel-Softmax Straight-Through (GS-ST), Score Function, and Alternative Path. Evaluated on stochastic biochemical systems exhibiting relaxation or oscillatory dynamics, GS-ST performs well in many settings but suffers from substantially increased variance in complex parameter regimes. In contrast, the Score Function and Alternative Path estimators demonstrate consistently lower variance and greater robustness, enabling accurate parameter inference. This work provides a practical framework and empirical guidance for differentiable modeling of discrete stochastic systems.

Abstract

Stochastic kinetic models are ubiquitous in physics, yet inferring their parameters from experimental data remains challenging. In deterministic models, parameter inference often relies on gradients, as they can be obtained efficiently through automatic differentiation. However, these tools cannot be directly applied to stochastic simulation algorithms (SSA) such as the Gillespie algorithm, since sampling from a discrete set of reactions introduces non-differentiable operations. In this work, we adopt three gradient estimators from machine learning for the Gillespie SSA: the Gumbel-Softmax Straight-Through (GS-ST) estimator, the Score Function estimator, and the Alternative Path estimator. We compare the properties of all estimators in two representative systems exhibiting relaxation or oscillatory dynamics, where the latter requires gradient estimation of time-dependent objective functions. We find that the GS-ST estimator mostly yields well-behaved gradient estimates, but exhibits diverging variance in challenging parameter regimes, resulting in unsuccessful parameter inference. In these cases, the other estimators provide more robust, lower variance gradients. Our results demonstrate that gradient-based parameter inference can be integrated effectively with the Gillespie SSA, with different estimators offering complementary advantages.

Problem

• stochastic kinetic models
• parameter inference
• gradient estimation
• Gillespie algorithm
• discrete reactions

Innovation

• gradient estimation
• stochastic kinetic models
• Gillespie algorithm
• parameter inference
• discrete stochastic simulation