Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics

2026-02-01

AI Summary

This work addresses the degradation of algorithmic stability with increasing dimensionality in sampling from high-dimensional multimodal distributions by proposing a dimension-independent sampling method based on preconditioned annealed Langevin dynamics. The approach approximates the target distribution as a Gaussian mixture model and introduces a preconditioning mechanism within a continuous-time framework, enabling a dimension-consistent theoretical analysis. By establishing spectral conditions linking the smoothed covariance to the component covariances of the mixture, the method rigorously demonstrates that preconditioning effectively suppresses error accumulation. Within a unified time scale, the algorithm achieves sampling accuracy independent of dimensionality. Numerical experiments confirm its stability and accuracy, showing strong agreement with theoretical predictions.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Designing algorithms that can explore multimodal target distributions accurately across successive refinements of an underlying high-dimensional problem is a central challenge in sampling. Annealed Langevin dynamics (ALD) is a widely used alternative to classical Langevin since it often yields much faster mixing on multimodal targets, but there is still a gap between this empirical success and existing theory: when, and under which design choices, can ALD be guaranteed to remain stable as dimension increases? In this paper, we help bridge this gap by providing a uniform-in-dimension analysis of continuous-time ALD for multimodal targets that can be well-approximated by Gaussian mixture models. Along an explicit annealing path obtained by progressively removing Gaussian smoothing of the target, we identify sufficient spectral conditions - linking smoothing covariance and the covariances of the Gaussian components of the mixture - under which ALD achieves a prescribed accuracy within a single, dimension-uniform time horizon. We then establish dimension-robustness to imperfect initialization and score approximation: under a misspecified-mixture score model, we derive explicit conditions showing that preconditioning the ALD algorithm with a sufficiently decaying spectrum is necessary to prevent error terms from accumulating across coordinates and destroying dimension-uniform control. Finally, numerical experiments illustrate and validate the theory.

Problem

• multimodal sampling
• high-dimensional stability
• annealed Langevin dynamics
• dimension-free analysis
• Gaussian mixture models

Innovation

• Preconditioned Annealed Langevin Dynamics
• Dimension-Free Sampling
• Multimodal Distributions
• Gaussian Mixture Models
• Spectral Conditions