density transformation

This work addresses the expressive power of geometric Gaussian approximations—specifically, pushing a standard Gaussian distribution onto complex target distributions (e.g., Bayesian posteriors) via diffeomorphisms or Riemannian exponential maps—and investigates whether a single diffeomorphism can uniformly approximate an entire family of distributions with high fidelity. Method: We establish rigorous theoretical guarantees for geometric Gaussian approximation, introducing a unified approximation framework for distribution families and proving equivalence between diffeomorphism-based and Riemannian exponential-map-based constructions. Results: We prove universality: any continuous probability distribution can be approximated to arbitrary precision by a Gaussian pushed forward via some diffeomorphism. Moreover, we construct an explicit, family-wide uniform approximation scheme and demonstrate formal equivalence between the two geometric approximation paradigms. These results provide a solid theoretical foundation and a novel design paradigm for efficient, interpretable probabilistic modeling in Bayesian inference.

Bayesian posteriors are often skewed, whereas mainstream deterministic approximations—such as Laplace’s method and variational Bayes—rely on symmetric densities (e.g., Gaussians), leading to systematic bias and reduced accuracy. Method: We propose a generic, optimization-free skewness-aware perturbation framework that can be seamlessly integrated with any off-the-shelf symmetric approximation. Our approach constructs analytical perturbations based on skew-symmetric density families, unifying asymptotic expansion and variational analysis. Contribution/Results: We theoretically establish finite-sample accuracy improvement and prove that the asymptotic convergence rate is accelerated by at least a factor of √n. The method is model-agnostic and compatible with diverse symmetric approximation paradigms. Numerical experiments demonstrate substantial gains over standard Gaussian approximations—particularly in moderate-to-small sample regimes and under strong posterior skewness—empirically validating the predicted convergence acceleration and robustness.

This paper addresses the scalability challenge of Bayesian posterior inference in reproducing kernel Hilbert spaces (RKHS). We propose a Langevin variational inference method grounded in Kosambi–Karhunen–Loève (K-L) truncation: the infinite-dimensional Langevin diffusion is projected onto the first $M$ K-L eigenfunctions, and a nonparametric variational family $mathcal{P}(mathbb{R}^M)$ is constructed in this finite-dimensional subspace to approximate the optimal posterior. We establish a theoretically guaranteed approximation error bound for convex Lipschitz negative log-likelihoods. Our framework unifies sparse variational Gaussian processes (SVGP) as a special case while overcoming their restrictive Gaussian process parametrization. The algorithm achieves computational complexity $mathcal{O}(M^3 + JM^2)$, where $J$ is the number of stochastic gradient steps, balancing theoretical optimality with practical scalability.

This work addresses the joint solution of forward (likelihood sampling) and inverse (posterior sampling) problems within the Bayesian framework. Methodologically, it introduces a novel invertible generative model that constructs a bijective mapping between parameter and observation spaces via stacked upper- and lower-triangular normalizing flows, and proposes a unified training objective for bidirectional conditional sampling—enabling end-to-end optimization of both forward simulation and posterior inference. Its key contribution lies in the first integration of invertible neural networks with Bayesian conditional generation, jointly modeling likelihood and posterior distributions without approximation bias inherent in traditional MCMC or variational inference. Across multiple numerical experiments, the model achieves high-fidelity simultaneous forward generation and inverse inference, significantly improving computational efficiency and statistical consistency of the Bayesian simulation–inference loop.

Bayesian neural networks (BNNs) suffer from parameterization-dependent approximate posteriors, violating reparameterization invariance and thereby decoupling parameter uncertainty from functional uncertainty—compromising uncertainty calibration and generalization. This work systematically investigates the issue within the linearized Laplace approximation framework. We propose the first reparameterization-invariant posterior sampling algorithm grounded in Riemannian diffusion processes; establish a geometric interpretation of linearized Laplace predictive success; and, for the first time, extend invariance guarantees from linearized predictions to predictions of the original nonlinear network. Experiments demonstrate substantial improvements in posterior approximation quality, yielding significantly more robust uncertainty calibration and generalization across multiple benchmarks.

This work addresses the challenges of efficiently sampling posterior distributions in Bayesian inversion when confronted with high-dimensional parameter spaces, sparse data, and strong noise, which hinder conventional dimensionality reduction techniques. The authors propose the α-likelihood informed subspace (α-LIS) method, which rigorously extends likelihood-informed subspace (LIS) theory to tempered posteriors with α ∈ [0,1], enabling the construction of a partially informed low-dimensional subspace for effective dimension reduction. By integrating data from multiple tempering levels and incorporating a gradient-free approximation strategy, the approach significantly enhances robustness and sampling efficiency in scenarios where gradients are unavailable or observations are highly noisy. Both theoretical analysis and numerical experiments demonstrate that near-optimal dimension reduction can be achieved with relatively small α values, yielding overall performance superior to traditional methods restricted to α = 1.

Implicit process priors, lacking closed-form expressions, pose significant challenges for Bayesian posterior inference in function space, and conventional Gaussian variational approximations often fail to capture asymmetric, heavy-tailed, or multimodal uncertainties. This work proposes a novel function-space variational inference approach that constructs a tractable prior approximation via a finite set of sampled functions and, for the first time, employs normalizing flows to flexibly model the combination weights, thereby replacing the restrictive Gaussian assumption. This method substantially enhances the expressiveness of the approximate posterior while preserving computational feasibility, effectively recovering complex posterior structures overlooked by Gaussian approximations and yielding more accurate uncertainty quantification across multiple tasks.

This work investigates the precise conditions under which Transformers can rigorously implement Bayesian posterior inference. Framed within measure-theoretic and Markov kernel formalisms, the authors develop an abstract hierarchy extending from single-layer Bayesian Transformers to multi-layer stacked architectures. Without imposing additional structural assumptions, they formally prove that when the internal update mechanism satisfies a joint distribution condition, the forward computation of the Transformer is equivalent to exact Bayesian posterior updating. The central contribution lies in establishing, for the first time, a rigorous mathematical correspondence between the softmax attention mechanism and Bayesian inference. Specifically, the update kernel induced by the Transformer block is shown to coincide almost everywhere with the true posterior distribution, and the key-value mapping generated by attention constitutes a valid probability distribution.

This work addresses the limitations of traditional Laplace approximations, which fail to capture posterior skewness, heavy tails, and narrow high-probability regions, as well as the computational expense of existing wrapped Gaussian methods that require evaluating geodesics, Christoffel symbols, or curvature tensors. By leveraging contrast function theory on a statistical manifold equipped with the Fisher–Rao metric and prior-induced geometry, the authors derive, for the first time, closed-form approximations of the exponential and logarithmic maps. This enables an efficient wrapped Gaussian approximation that avoids costly geometric computations. The proposed method substantially reduces computational complexity while accurately capturing complex posterior geometries across diverse models, achieving speedups of several orders of magnitude over current state-of-the-art approaches.

This work addresses the systematic underestimation of posterior functionals—such as expectations and variances—in structured mean-field variational inference, which arises from neglected cross-block dependencies whose geometric origin remains unclear. We introduce a differential-geometric framework based on the tangent space induced by the variational family and establish, for the first time, a precise link between functional bias and tangent space geometry. Specifically, we show that the component orthogonal to the tangent space dominates the first-order bias, whereas the component within the tangent space—spanned by block-additive functions—contributes only to second-order bias. By integrating local asymptotic normality with functional asymptotic expansions, we derive an explicit expression for the bias, offering a geometric explanation for the distortion of cross-block dependencies inherent in mean-field approximations.