Nesterov acceleration in optimizing over probability measures

2026-07-24

AI Summary

Extending accelerated optimization methods to the space of probability measures poses significant challenges due to the difficulty in defining momentum and implementing algorithms numerically. This work proposes two complementary lifting frameworks—based on Hamiltonian phase-space embedding and reproducing kernel Hilbert space embedding—to recover linear structures amenable to theoretical analysis and to derive implementable particle dynamics. Building upon these frameworks, Heavy-ball and Nesterov-type accelerated algorithms are constructed for the Wasserstein space, accompanied by non-asymptotic convergence guarantees. The resulting methods achieve iteration complexities matching those of Nesterov’s scheme in Euclidean space, and the influence of the number of particles on approximation accuracy is rigorously quantified.

Abstract

Optimization over probability measures has become an increasingly important paradigm in modern machine learning, scientific computing, and uncertainty quantification. Motivated by Nesterov's accelerated gradient method in Euclidean space, we develop Heavy-ball and Nesterov acceleration methods over the probability measure space $\mathcal{P}_2$ and establish non-asymptotic convergence guarantees that match their Euclidean counterparts. In particular, we derive convergence rates with respect to both the number of iterations and the number of particles used to represent the underlying probability distributions. Extending accelerated optimization from Euclidean space to probability measures is challenging. The natural notion of momentum requires concepts such as tangent bundles of the set of probability space and they are hard to operate numerically. To overcome these difficulties, we introduce two complementary lifting procedures. The first lifts probability measures to phase space through a Hamiltonian formulation, introducing momentum variables into the dynamics. The second lifts probability measures to a common Hilbert space, restoring the linear structure required for convergence analysis while simultaneously yielding executable particle dynamics. Together, these two complementary lifting procedures provide a systematic methodology for designing, analyzing, and implementing momentum-based accelerated optimization methods over probability measure spaces.

Problem

• accelerated optimization
• probability measures
• Nesterov acceleration
• momentum
• non-Euclidean spaces

Innovation

• Nesterov acceleration
• probability measures
• Wasserstein space
• momentum methods
• particle dynamics