Bayesian Computation in Deep Learning

2025-02-25

AI Summary

This work addresses the core challenge of accurately inferring high-dimensional, non-convex posterior distributions in Bayesian neural networks and deep generative models. To this end, it systematically establishes the first comprehensive methodology framework for Bayesian approximate inference tailored to deep learning. The framework unifies variational inference, Markov chain Monte Carlo (including stochastic gradient samplers), Laplace approximation, and probabilistic programming into a coherent classification taxonomy and practical paradigm—thereby bridging Bayesian computation with modern deep architectures. The proposed methods substantially improve both posterior approximation accuracy and computational efficiency. Empirically validated across diverse deep Bayesian models, they deliver a theoretically principled yet engineering-practical inference toolkit for trustworthy AI systems.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Reasoning under Uncertainty: Probabilistic Programming
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

This review paper is intended for the 2nd edition of the Handbook of Markov chain Monte Carlo.We provide an introduction to approximate inference techniques as Bayesian computation methods applied to deep learning models. We organize the chapter by presenting popular computational methods for (1) Bayesian neural networks and (2) deep generative models, explaining their unique challenges in posterior inference as well as the solutions.

Problem

• Bayesian computation in deep learning
• Challenges in posterior inference
• Solutions for Bayesian neural networks

Innovation

• Bayesian computation in deep learning
• Approximate inference techniques
• Posterior inference solutions