AutoBayes: A Compositional Framework for Generalized Variational Inference

2025-03-24

AI Summary

Generalized variational inference suffers from tight coupling between modeling and optimization, hindering modularity and composability. Method: We propose the first composable, unified framework for variational inference. Leveraging a newly discovered chain rule—akin to reverse-mode automatic differentiation—that governs the interplay between Bayesian inference and variational objectives, we design composable operators that decouple model structure, inverse modeling, local loss binding, and parameter exposure. Further, we introduce a statistical game-theoretic perspective to enable localized optimization. Contribution/Results: Experiments across canonical Bayesian models demonstrate that our framework significantly improves modularity, interpretability, and construction efficiency of variational inference. It supports flexible, plug-and-play assembly of arbitrary subcomponents and end-to-end optimization, establishing a novel paradigm for complex probabilistic modeling.

Technology Category

Reasoning under Uncertainty: Probabilistic InferenceMachine Learning: Probabilistic Circuits and Graphical ModelsComputer Vision: Learning & Optimization for CV

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationEconomics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

We introduce a new compositional framework for generalized variational inference, clarifying the different parts of a model, how they interact, and how they compose. We explain that both exact Bayesian inference and the loss functions typical of variational inference (such as variational free energy and its generalizations) satisfy chain rules akin to that of reverse-mode automatic differentiation, and we advocate for exploiting this to build and optimize models accordingly. To this end, we construct a series of compositional tools: for building models; for constructing their inversions; for attaching local loss functions; and for exposing parameters. Finally, we explain how the resulting parameterized statistical games may be optimized locally, too. We illustrate our framework with a number of classic examples, pointing to new areas of extensibility that are revealed.

Problem

• Develops a compositional framework for generalized variational inference
• Explains chain rules in Bayesian and variational inference
• Provides tools for model building, inversion, and optimization

Innovation

• Compositional framework for variational inference
• Chain rules like automatic differentiation
• Local optimization of parameterized statistical games