Accelerating Particle-based Energetic Variational Inference

2025-04-04

AI Summary

To address the high computational cost, limited robustness, and suboptimal accuracy of particle-based variational inference (ParVI), this paper proposes EVI-Im—a novel energy-based variational inference method with implicit discretization. EVI-Im is the first to integrate energy quadratization (EQ) with operator splitting, establishing a “discretize-then-variational” framework; it employs implicit Euler discretization to preserve the gradient-flow structure of the underlying dynamics. Furthermore, it introduces an efficient discrete strategy that avoids redundant computation of inter-particle interactions. Experiments across multiple benchmark tasks demonstrate that EVI-Im achieves up to 3.2× speedup over state-of-the-art ParVI methods, while significantly improving sampling robustness and distributional approximation accuracy—evidenced by lower KL divergence. By jointly ensuring computational efficiency, numerical stability, and structural fidelity to the target gradient flow, EVI-Im establishes a new paradigm for high-dimensional Bayesian inference via structure-preserving variational particle dynamics.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Probabilistic Circuits and Graphical Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

In this work, we propose a novel particle-based variational inference (ParVI) method that accelerates the EVI-Im. Inspired by energy quadratization (EQ) and operator splitting techniques for gradient flows, our approach efficiently drives particles towards the target distribution. Unlike EVI-Im, which employs the implicit Euler method to solve variational-preserving particle dynamics for minimizing the KL divergence, derived using a"discretize-then-variational"approach, the proposed algorithm avoids repeated evaluation of inter-particle interaction terms, significantly reducing computational cost. The framework is also extensible to other gradient-based sampling techniques. Through several numerical experiments, we demonstrate that our method outperforms existing ParVI approaches in efficiency, robustness, and accuracy.

Problem

• Accelerates particle-based variational inference for efficiency
• Reduces computational cost by avoiding repeated interaction terms
• Extends framework to gradient-based sampling techniques

Innovation

• Uses energy quadratization for efficient particle dynamics
• Avoids repeated inter-particle interaction evaluations
• Extends to gradient-based sampling techniques