Making Recursive Bayesian Inference Robust

2026-06-06

AI Summary

This study addresses the challenge of inaccurate estimation in recursive Bayesian inference when significant inter-stage shifts occur in the posterior distribution. To overcome this limitation, the authors propose and implement a Parallel Annealed Prior-Proposal Recursive Bayesian (PPP-RB) method that integrates recursive Bayesian updating, parallel computation, and an annealing mechanism inspired by Metropolis-coupled Markov chain Monte Carlo. PPP-RB maintains accurate approximation of the true posterior across multiple updating stages, effectively resolving the failure of existing prior-proposal recursive Bayesian (PP-RB) approaches under substantial posterior shifts. The method provides theoretical guarantees for correctness of the target posterior while substantially improving computational efficiency. Empirical evaluations on earthquake count data and North Atlantic sea surface salinity measurements demonstrate that PPP-RB yields a higher effective sample size per unit time compared to both PP-RB and standard MCMC algorithms.

Abstract

While Bayesian inference has become increasingly popular with advances in computational resources, its algorithms can be computationally prohibitive and may not scale with large datasets. This has led to growing interest in alternative algorithms, such as approximation methods and variants of Markov chain Monte Carlo. Among these approaches, prior proposal-recursive Bayesian (PP-RB) inference facilitates scalable Bayesian computation by recursively updating the posterior distribution across stages and utilizing parallel computing resources. While the well-known ``degeneracy'' issue in PP-RB has been studied, another limitation that PP-RB can yield incorrect inferences when posterior distributions shift substantially between stages has remained unsolved. To address this, we propose parallel-tempered prior proposal-recursive Bayesian (PPP-RB) inference, which extends PP-RB by leveraging the key idea underlying Metropolis-coupled Markov chain Monte Carlo. We show both theoretically and empirically that PPP-RB targets the true posterior distribution. We illustrate PPP-RB through numerical studies and real data analysis in application to earthquake count data and sea surface salinity in the North Atlantic region. In these applications, we compare PPP-RB with PP-RB and a standard MCMC, demonstrating that PPP-RB is more efficient in terms of effective sample size per elapsed time.

Problem

• Recursive Bayesian inference
• Posterior shift
• Degeneracy
• Scalable Bayesian computation
• Incorrect inference

Innovation

• recursive Bayesian inference
• parallel tempering
• Metropolis-coupled MCMC
• posterior degeneracy
• scalable Bayesian computation