Likelihood-informed dimension reduction across tempered Bayesian posteriors

2026-05-20

AI Summary

This work addresses the challenges of efficiently sampling posterior distributions in Bayesian inversion when confronted with high-dimensional parameter spaces, sparse data, and strong noise, which hinder conventional dimensionality reduction techniques. The authors propose the α-likelihood informed subspace (α-LIS) method, which rigorously extends likelihood-informed subspace (LIS) theory to tempered posteriors with α ∈ [0,1], enabling the construction of a partially informed low-dimensional subspace for effective dimension reduction. By integrating data from multiple tempering levels and incorporating a gradient-free approximation strategy, the approach significantly enhances robustness and sampling efficiency in scenarios where gradients are unavailable or observations are highly noisy. Both theoretical analysis and numerical experiments demonstrate that near-optimal dimension reduction can be achieved with relatively small α values, yielding overall performance superior to traditional methods restricted to α = 1.

Abstract

Scientific computer simulations cannot represent all scales in realistic applications. To bridge this model-data gap, parameters are injected into models and constrained with noisy data using Bayesian inversion. To reduce the number of simulator evaluations, which can be 10^5 or more, modern approaches employ dimension reduction in conjunction with emulation of the forward map (that contains the simulator). Due to scarcity of model evaluations and data, this dimension reduction becomes very important for posterior sampling performance. Recent work on likelihood-informed subspaces (LIS) truncates to informative directions by optimizing bounds on information loss, and though mathematically well-adapted to sampling, they are often restrictive in practice. In this work, we provably generalize this methodology to facilitate application to $α$-tempered (i.e., annealed, power-posterior) distributions for $α$ in [0,1]. We provide theory to build partially-informed spaces termed $α$-LIS. We show how $α$ < 1 can often produce near-optimal spaces. In addition, we focus on applying $α$-LIS to practical cases, where the available data is severely limited and noisy. We propose and test extensions for utilizing data from the entire sequence of distributions $α$_0 < ... < $α$_k, and use simple approximations of model gradients so that our approach can be used for emulation of forward maps for chaotic or stochastic systems where derivatives are unavailable or uninformative due to noise. In experiments, our accumulated approach is much more robust to these challenging circumstances than the theoretically optimal $α$ = 1.

Problem

• dimension reduction
• Bayesian inversion
• likelihood-informed subspaces
• tempered posteriors
• forward map emulation

Innovation

• likelihood-informed subspace
• dimension reduction
• tempered posterior
• Bayesian inversion
• emulation