Adaptive reduced tempering For Bayesian inverse problems and rare event simulation

2024-10-24

AI Summary

Bayesian inverse problems and rare-event simulation under expensive likelihood evaluations face the challenge that the posterior concentrates in sparse, a priori unknown regions of the parameter space, making it difficult for conventional methods to balance accuracy and efficiency. Method: We propose an adaptive sequential Monte Carlo (SMC) algorithm that jointly optimizes a surrogate model—based on reduced-basis approximation for elliptic PDE solutions—and a temperature annealing schedule. Our method introduces a novel posterior-entropy-driven adaptive inverse-temperature selection mechanism, dynamically coupling surrogate error estimation with temperature scheduling to enable snapshot-wise accurate likelihood evaluation and uncertainty-aware surrogate refinement in critical regions. Contribution/Results: Theoretical analysis guarantees convergence. Numerical experiments on PDE inverse problems demonstrate approximately tenfold reduction in computational cost while preserving posterior statistical accuracy.

Technology Category

Reasoning under Uncertainty: Stochastic OptimizationMachine Learning: Calibration & Uncertainty QuantificationSearch and Optimization: Sampling/Simulation-based Search

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

This work proposes an adaptive sequential Monte Carlo sampling algorithm for solving inverse Bayesian problems in a context where a (costly) likelihood evaluation can be approximated by a surrogate, constructed from previous evaluations of the true likelihood. A rough error estimation of the obtained surrogates is required. The method is based on an adaptive sequential Monte-Carlo (SMC) simulation that jointly adapts the likelihood approximations and a standard tempering scheme of the target posterior distribution. This algorithm is well-suited to cases where the posterior is concentrated in a rare and unknown region of the prior. It is also suitable for solving low-temperature and rare-event simulation problems. The main contribution is to propose an entropy criteria that associates to the accuracy of the current surrogate a maximum inverse temperature for the likelihood approximation. The latter is used to sample a so-called snapshot, perform an exact likelihood evaluation, and update the surrogate and its error quantification. Some consistency results are presented in an idealized framework of the proposed algorithm. Our numerical experiments use in particular a reduced basis approach to construct approximate parametric solutions of a partially observed solution of an elliptic Partial Differential Equation. They demonstrate the convergence of the algorithm and show a significant cost reduction (close to a factor $10$) for comparable accuracy.

Problem

• Adaptive SMC algorithm for costly Bayesian inverse problems
• Uses surrogate models to approximate expensive likelihood evaluations
• Targets rare event simulation and concentrated posterior distributions

Innovation

• Adaptive SMC with surrogate likelihood approximation
• Entropy criterion linking surrogate accuracy to temperature
• Reduced basis approach for cost-effective PDE solutions