Learn-Then-Differentiate Gradient Estimation

2026-09-29

AI Summary

This study addresses the absence of a unified theoretical framework and accuracy guarantees for gradient estimation via simulation output. To this end, it proposes the LTD (Learning to Differentiate) unified framework. Methodologically, by leveraging a differential interpretation of probability measure learning under weighted representations, this work translates model fitting accuracy into error bounds for gradient estimation. It further provides a unified convergence rate analysis for diverse methods, including kernel regression, local polynomial regression, kernel ridge regression, and smoothed neural networks. The primary contribution lies in achieving convergence rates comparable to standard Monte Carlo methods under smoothness conditions, thereby establishing a common theoretical foundation and rigorous accuracy guarantees for gradient estimation across multiple classes of learning approaches.

Abstract

Learn-then-differentiate (LTD) estimates gradients by fitting a model to simulation outputs and differentiating it. We develop a unified framework explaining what LTD differentiates and how accurately it estimates gradients. For models with a weighted representation, LTD differentiates a learned representation of the underlying probability measure. We then show how accuracy guarantees for fitted models translate into guarantees for gradients and higher-order derivatives, with rates approaching the standard Monte Carlo rate under suitable smoothness conditions. The framework recovers established results for kernel regression, local polynomial regression, and kernel ridge regression, and yields further guarantees for multiple kernel learning and smooth neural networks. These results provide a common foundation for understanding and analyzing LTD across learning methods.

Problem

• gradient estimation
• learn-then-differentiate
• simulation-based inference
• theoretical framework

Innovation

• Learn-then-Differentiate
• Gradient Estimation
• Unified Framework
• Monte Carlo Rate
• Kernel Methods