multifidelity simulation-based inference

Addressing the prohibitively high computational cost of parameter inference with high-fidelity simulators, this paper proposes the Multi-Fidelity Neural Posterior Estimation (MF-NPE) framework—the first to jointly integrate transfer learning and active learning into Bayesian inference. MF-NPE leverages inexpensive, low-fidelity simulation data to guide posterior distribution learning for high-fidelity simulators, while actively selecting the most informative high-fidelity simulation queries to minimize their evaluation count. Evaluated on three benchmark tasks, MF-NPE achieves inference accuracy comparable to state-of-the-art methods while reducing high-fidelity simulator calls by up to two orders of magnitude, substantially improving computational efficiency. Its core contributions lie in enabling cross-fidelity knowledge transfer and adaptive optimization of simulation resources.

Multi-fidelity modeling faces the core challenge of scarce and expensive high-fidelity (HF) data versus abundant yet biased low-fidelity (LF) data. This paper proposes a generic three-stage framework: (1) using a deterministic LF model as the foundation, (2) enabling cross-fidelity knowledge transfer via transfer learning, and (3) quantifying residual uncertainty through Bayesian residual modeling. It is the first to reveal the complementary expressive power between transfer learning and Bayesian modeling, unifying treatment of both noisy and noise-free multi-fidelity settings while substantially simplifying existing approaches. Technically, it supports flexible combinations—e.g., kernel ridge regression or deep neural networks for LF modeling, and Gaussian processes or Bayesian neural networks for HF modeling—under a staged training strategy. Extensive benchmark experiments demonstrate significant improvements over state-of-the-art methods in prediction accuracy, uncertainty calibration, and computational efficiency, achieving both theoretical rigor and engineering practicality.

Likelihood-free and gradient-free parameter calibration in black-box simulators poses significant challenges for Bayesian inference. Method: This paper introduces the first simulation-based, fully amortized, gradient-free, and parallelizable neural Bayesian inference framework, accompanied by the open-source PyTorch package SBI. The framework unifies neural posterior estimation (NPE), neural likelihood estimation (NLE), neural ratio estimation (NRE), and mixture density networks (MDNs), integrating Monte Carlo sampling, Bayesian optimization, and simulation scheduling into a modular, end-to-end workflow with production-ready defaults and comprehensive diagnostic tools. Contribution/Results: Evaluated across physics, biology, and astronomy, SBI substantially lowers the barrier to simulation-based inference, accelerates posterior estimation by multiple-fold, and achieves state-of-the-art reusability and scalability.

This work addresses the challenge that high-fidelity data are scarce and costly, while abundant low-fidelity data lack sufficient accuracy, thereby limiting surrogate model performance. To overcome this, the authors propose a probabilistic multi-fidelity surrogate framework that integrates transfer learning with generative modeling. Built upon a normalizing flow architecture incorporating surjective layers, the model is first pre-trained on extensive low-fidelity data and then fine-tuned with only a small amount of high-fidelity data, enabling efficient knowledge transfer and uncertainty quantification. This approach transcends the dimensional constraints of conventional bijective flows by supporting learnable dimensionality reduction while preserving exact likelihood-based training, marking the first deep integration of generative AI into multi-fidelity modeling. Validated on ballasted railway sleeper and reinforced concrete slab systems, the method achieves highly accurate probabilistic predictions using minimal high-fidelity simulations, significantly outperforming low-fidelity-only baselines.

For high-dimensional simulator-based models with intractable likelihoods, this paper proposes an efficient and stable Sequential Neural Posterior Estimation (SNPE) method. The approach employs conditional neural density estimation within a sequential simulation framework, augmented by an adaptive calibration kernel mechanism—novelly introduced herein—to dynamically adjust kernel weights during inference. To further enhance stability and accelerate convergence, we integrate importance-weighted gradient variance reduction with Monte Carlo loss optimization. This combination effectively mitigates the inference bottlenecks inherent in high-dimensional settings while preserving posterior approximation accuracy. Extensive experiments on multiple benchmark simulators and real-world high-dimensional datasets demonstrate that our method achieves over a two-fold speedup in training time and reduces posterior approximation error by more than 30% compared to standard SNPE and other state-of-the-art approaches.

This work addresses the issue of overconfidence in statistical inference arising from machine learning approximations in scientific simulations, which can compromise result reliability. To mitigate this, the authors propose two complementary approaches: first, a “balanced” regularization strategy that explicitly suppresses model overconfidence; and second, a simulation-aware Bayesian neural network prior that naturally alleviates overconfidence without additional regularization, even in small-sample regimes. By integrating neural ratio estimation with uncertainty quantification techniques, the proposed methods significantly improve inference calibration, yielding posterior estimates that are either closer to the ground truth or conservatively biased. This enhanced calibration strengthens the credibility of simulation-based inference in scientific applications.

This work addresses the challenge of parameter inference in stochastic processes, where conventional simulation-based inference methods suffer from computationally expensive likelihood evaluations and struggle to balance surrogate model accuracy against simulation cost under limited data. Breaking from the black-box assumption, this study introduces a novel approach that incorporates exact score information and a loss-gradient–based adaptive weighting mechanism into a probabilistic classification framework for neural likelihood surrogates, optimizing binary cross-entropy loss. The proposed method substantially enhances both the efficiency and accuracy of the surrogate model. Across multiple stochastic process benchmarks, it achieves downstream inference performance equivalent to using ten times more training data while requiring only 1.1× the original training time, effectively alleviating the data–cost trade-off bottleneck.

This study addresses the challenges of efficiency and accuracy in parameter estimation—a key inverse problem in complex statistical modeling where data are accessible only through simulation. The authors propose a frequentist approach based on a single summary network that directly learns parameter estimators from simulated data by minimizing the mean squared error between true parameters and the network’s output summaries. The method employs a branched network architecture with a collapse layer, theoretically designed to balance estimation accuracy under finite samples, robustness to contaminated data, and the ability to automatically approximate algorithmically reconstructed data. Experiments on genetic data simulations demonstrate that the framework successfully replicates the performance of the EM algorithm while exhibiting superior accuracy, robustness, and algorithmic approximation capability.

This work addresses the high computational cost incurred by repeated simulations in hierarchical simulation-based inference by proposing Tokenized Flow Matching Posterior Estimation (TFMPE). The method integrates likelihood factorization with neural surrogate models, enabling training from single-site simulations and introducing, for the first time, tokenized flow matching into hierarchical Bayesian inference. This approach facilitates efficient amortized posterior estimation under functional observations. Experiments on a newly established hierarchical SBI benchmark, as well as on epidemiological and computational fluid dynamics models, demonstrate that TFMPE substantially reduces simulation overhead while yielding well-calibrated posterior distributions.

Existing neural posterior estimation methods struggle to handle mixed parameter spaces containing both discrete and continuous variables, limiting their applicability in complex scientific simulations. This work presents the first extension of simulation-based inference (SBI) to such hybrid spaces by introducing a unified joint inference framework: it models discrete parameters via an autoregressive classifier and continuous parameters through a generative model, with both components trained jointly under a single objective. Implemented within the sbi toolkit and accompanied by posterior calibration diagnostics, the proposed method yields accurate and well-calibrated posterior estimates across multiple analytically tractable toy models and realistic scientific simulators, substantially enhancing the practicality and reliability of inference in mixed-parameter settings.