mc dropout estimation

Monte Carlo Dropout (MCD) is widely adopted as a lightweight approximation to Bayesian uncertainty estimation, yet its reliability—particularly in modeling epistemic and aleatoric uncertainty—lacks systematic empirical validation. Method: This work conducts the first rigorous comparative evaluation of MCD against gold-standard Bayesian methods—Gaussian processes and fully Bayesian neural networks—across extrapolation and interpolation regimes, with controlled experimental design and comprehensive uncertainty calibration and discrimination metrics. Results: MCD severely underestimates uncertainty in extrapolation regions, exhibiting poor calibration and low discriminative power; critically, it fails to replicate the theoretically grounded increase in epistemic uncertainty with data sparsity. These findings expose a systemic failure of MCD as a Bayesian approximation, challenging its default deployment in high-stakes, risk-sensitive applications such as autonomous driving and medical diagnosis. The study establishes an empirical benchmark and theoretical caution for uncertainty quantification, informing both method selection and future improvements in approximate Bayesian inference.

This study addresses the uncontrollable error and computational inefficiency arising from fixed sampling budgets in Monte Carlo inference for Bayesian neural networks (BNNs). To this end, it introduces confidence sequence theory into BNN inference for the first time, proposing an adaptive sampling termination algorithm. This method dynamically determines the required number of samples based on specific decision-making objectives, such as classification or distribution approximation, enabling on-demand stopping with rigorous statistical guarantees. Experimental results demonstrate that the proposed mechanism intelligently allocates computational resources by automatically increasing the sample count for ambiguous inputs. Consequently, it significantly reduces overall inference latency while preserving decision reliability.

This work addresses the issue of overconfidence in statistical inference arising from machine learning approximations in scientific simulations, which can compromise result reliability. To mitigate this, the authors propose two complementary approaches: first, a “balanced” regularization strategy that explicitly suppresses model overconfidence; and second, a simulation-aware Bayesian neural network prior that naturally alleviates overconfidence without additional regularization, even in small-sample regimes. By integrating neural ratio estimation with uncertainty quantification techniques, the proposed methods significantly improve inference calibration, yielding posterior estimates that are either closer to the ground truth or conservatively biased. This enhanced calibration strengthens the credibility of simulation-based inference in scientific applications.

Bayesian neural networks (BNNs) rely on posterior sampling for uncertainty quantification and out-of-distribution (OOD) robustness, yet stochastic gradient Markov chain Monte Carlo (SGMCMC) methods often suffer from insufficient sample diversity, leading to biased posterior estimates. To address this, we propose a parameter-expansion reparameterization strategy: the weight matrix is factorized into a product of lower-rank matrices, enabling enhanced trajectory exploration and faster mixing of SGMCMC chains—without increasing computational cost, temperature scaling, or multi-chain parallelism. Our approach unifies matrix factorization, Langevin dynamics, and Bayesian inference, with theoretical guarantees of posterior consistency. Experiments on image classification demonstrate substantial improvements over standard SGMCMC and Hamiltonian Monte Carlo: higher OOD detection accuracy, greater sample diversity, broader coverage of the loss landscape, and unchanged inference overhead.

Likelihood-free and gradient-free parameter calibration in black-box simulators poses significant challenges for Bayesian inference. Method: This paper introduces the first simulation-based, fully amortized, gradient-free, and parallelizable neural Bayesian inference framework, accompanied by the open-source PyTorch package SBI. The framework unifies neural posterior estimation (NPE), neural likelihood estimation (NLE), neural ratio estimation (NRE), and mixture density networks (MDNs), integrating Monte Carlo sampling, Bayesian optimization, and simulation scheduling into a modular, end-to-end workflow with production-ready defaults and comprehensive diagnostic tools. Contribution/Results: Evaluated across physics, biology, and astronomy, SBI substantially lowers the barrier to simulation-based inference, accelerates posterior estimation by multiple-fold, and achieves state-of-the-art reusability and scalability.

This work proposes a novel approach to simulation-based inference by integrating large language model–driven program synthesis, enabling joint inference of both model structure and parameters—a capability lacking in traditional methods that rely on fixed, pre-specified simulator architectures. By automatically generating and iteratively refining candidate simulator programs from natural language descriptions, the method transcends rigid modeling assumptions and facilitates the discovery of plausible models directly from open-ended prompts. Empirical evaluations across diverse domains—including deterministic dynamical systems, stochastic epidemic models, and gravitational lensing image analysis—demonstrate its ability to accurately identify data-supported model families, revealing the interplay between the informativeness of observed data and the identifiability of candidate models.

This work addresses inverse problems in science and engineering—such as parameter inference and detector response unfolding—by proposing a unified simulation-based inference (SBI) framework that systematically integrates Bayesian and frequentist perspectives. Leveraging machine learning techniques, including neural posterior estimation and neural likelihood estimation, the framework enables efficient and general-purpose parameter inference, with extensions to empirical Bayes and unfolding tasks. The paper provides a comprehensive review of SBI methodologies and their application paradigms, while also offering a thorough analysis of validation strategies and inherent limitations. By clarifying best practices and pitfalls, this study advances the reliable deployment and innovative application of SBI in scientific domains.

This work addresses the challenge of accurately quantifying uncertainty and tuning hyperparameters in stochastic gradient Markov chain Monte Carlo methods under large-batch settings or model misspecification. The authors propose a novel discrete-time approximation framework applicable to both momentum and non-momentum variants of stochastic gradient (Langevin) dynamics (SG(L)D). This framework enables precise prediction of the stationary covariance, iterate-averaged covariance, and integrated autocorrelation time. Notably, it establishes the first non-asymptotic, quantitative error bounds in a discrete-time setting, yielding a high-fidelity characterization of SG(L)D behavior in complex scenarios. By integrating β-divergence–based robust inference with covariance estimation, the method consistently outperforms existing tuning strategies across diverse models and data distributions, maintaining superior uncertainty quantification even under significant model misspecification.

This study addresses the high computational cost and insufficient reliability guarantees of neural simulation-based inference by proposing a hybrid inference framework incorporating semiparametric formulations. By introducing two mixture strategies, including latent classes, the method effectively balances inference sensitivity with computational efficiency while preserving the statistical reliability of parametric models. Experimental results demonstrate that the proposed approach significantly reduces computational overhead at the expense of only marginal sensitivity loss. It supports both offline analysis and future trigger-level real-time applications, providing a theoretically grounded and practically valuable solution for efficient and robust statistical inference.

This study addresses the challenge of jointly modeling calibration and control parameters in computer model calibration, where the distribution of calibration parameters is unknown while that of control parameters is known. To tackle this issue, the authors propose a nonparametric Bayesian calibration method based on measure decomposition. The approach preserves the known marginal distribution of the control parameters while employing stochastic process modeling and Bayesian inference to construct a posterior distribution over the input space that aligns with field observations. Notably, this work is the first within a nonparametric calibration framework to explicitly maintain the prior distributional properties of the control parameters, thereby substantially enhancing the physical consistency and scientific credibility of the calibration results.