Statistical Inference on Gradient Flows

2026-05-31

AI Summary

Existing methods struggle to perform full-path statistical inference for gradient flow optimization trajectories, particularly lacking valid uncertainty quantification when stopping times are data-dependent or the path diverges. This work establishes a time-uniform statistical inference theory for gradient flows by modeling the deviation between empirical and population gradient flows as a continuous Gaussian process indexed over the non-negative real line. It presents the first uniform central limit theorem applicable across the entire optimization trajectory. Building on this foundation, the paper introduces an algorithm-aware covariance estimator that requires neither matrix inversion, resampling, nor data splitting, and which converges uniformly over time. The resulting confidence bands achieve asymptotically valid coverage, offering a theoretically rigorous and practically useful tool for path-level uncertainty quantification in gradient-based algorithms.

Abstract

Gradient-based algorithms are central to modern statistical estimation, yet their statistical analysis is often restricted to fixed-time behavior, such as convergence to a population target or fluctuations at a prescribed iteration. In many applications, however, uncertainty quantification is needed along the entire optimization path, especially when the stopping time is data-dependent or divergent. In this paper, we develop a theory for time-uniform statistical inference on gradient flows arising from empirical risk minimization. We prove a uniform central limit theorem that characterizes the deviation between empirical and population gradient flows as a continuous-time Gaussian process over the entire nonnegative real line. Building on this result, we introduce an algorithm-aware covariance estimator that evolves jointly with the gradient flow and avoids matrix inversion, resampling, or sample splitting. We show that the covariance estimator is uniformly consistent over time and use it to construct confidence intervals for the target parameter with asymptotically valid coverage. Our results connect optimization dynamics with statistical inference and provide practical tools for uncertainty quantification in gradient-based methods.

Problem

• statistical inference
• gradient flows
• uncertainty quantification
• time-uniform inference
• empirical risk minimization

Innovation

• gradient flows
• uniform central limit theorem
• time-uniform inference
• covariance estimation
• uncertainty quantification