Likelihood Level Adapted Estimation of Marginal Likelihood for Bayesian Model Selection

2025-03-29

AI Summary

In computational mechanics, Bayesian model selection requires efficient estimation of high-dimensional marginal likelihoods (model evidences), a computationally challenging task. To address this, we propose a novel framework based on adaptive hierarchical sampling over iso-likelihood contours: the high-dimensional integral is reduced to a one-dimensional adaptive numerical integration, synergistically integrating importance sampling, stratified sampling, and MCMC techniques, augmented by Karhunen–Loève expansion for dimensionality reduction. The method significantly improves both accuracy and convergence rate of marginal likelihood estimation for high-dimensional, complex models. Four benchmark case studies validate its efficacy: (i) marginal likelihood estimation error <1% against analytical solutions; (ii) improved model selection accuracy for an 11-story base-isolated structure; (iii) faster convergence and enhanced robustness over MultiNest in cylinder flow inference; and (iv) superior performance in a 100-dimensional heterogeneous heat conduction inverse problem.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In computational mechanics, multiple models are often present to describe a physical system. While Bayesian model selection is a helpful tool to compare these models using measurement data, it requires the computationally expensive estimation of a multidimensional integral -- known as the marginal likelihood or as the model evidence ( extit{i.e.}, the probability of observing the measured data given the model). This study presents efficient approaches for estimating this marginal likelihood by transforming it into a one-dimensional integral that is subsequently evaluated using a quadrature rule at multiple adaptively-chosen iso-likelihood contour levels. Three different algorithms are proposed to estimate the probability mass at each adapted likelihood level using samples from importance sampling, stratified sampling, and Markov chain Monte Carlo sampling, respectively. The proposed approach is illustrated through four numerical examples. The first example validates the algorithms against a known exact marginal likelihood. The second example uses an 11-story building subjected to an earthquake excitation with an uncertain hysteretic base isolation layer with two models to describe the isolation layer behavior. The third example considers flow past a cylinder when the inlet velocity is uncertain. Based on these examples, the method with stratified sampling is by far the most accurate and efficient method for complex model behavior in low dimension. In the fourth example, the proposed approach is applied to heat conduction in an inhomogeneous plate with uncertain thermal conductivity modeled through a 100 degree-of-freedom Karhunen-Lo`{e}ve expansion. The results indicate that MultiNest cannot efficiently handle the high-dimensional parameter space, whereas the proposed MCMC-based method more accurately and efficiently explores the parameter space.

Problem

• Efficient estimation of marginal likelihood for Bayesian model selection
• Transforming multidimensional integral into one-dimensional integral
• Comparing sampling methods for accuracy in high-dimensional spaces

Innovation

• Transforms marginal likelihood into 1D integral
• Uses adaptive iso-likelihood contour quadrature
• Compares sampling methods for probability estimation