Fast and Robust Simulation-Based Inference With Optimization Monte Carlo

2025-11-17

AI Summary

Bayesian parameter inference for complex stochastic simulators suffers from prohibitive computational cost due to intractable likelihoods—especially in high-dimensional parameter spaces or when simulator outputs are information-poor. To address this, we propose OptiBayes: a novel framework that reformulates stochastic simulation as a differentiable deterministic optimization problem. By leveraging gradient-based optimization, OptiBayes rapidly identifies high-posterior-density regions while substantially suppressing wasteful simulations in low-probability areas. Built upon an optimization–Monte Carlo hybrid paradigm and implemented end-to-end with JAX, it enables full vectorization and differentiability throughout the inference pipeline. Experiments demonstrate that OptiBayes matches or exceeds state-of-the-art methods in estimation accuracy while reducing runtime by one to two orders of magnitude. It further exhibits strong scalability, robustness to noise and model misspecification, and exceptional adaptability to weakly informative simulator outputs.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Bayesian parameter inference for complex stochastic simulators is challenging due to intractable likelihood functions. Existing simulation-based inference methods often require large number of simulations and become costly to use in high-dimensional parameter spaces or in problems with partially uninformative outputs. We propose a new method for differentiable simulators that delivers accurate posterior inference with substantially reduced runtimes. Building on the Optimization Monte Carlo framework, our approach reformulates stochastic simulation as deterministic optimization problems. Gradient-based methods are then applied to efficiently navigate toward high-density posterior regions and avoid wasteful simulations in low-probability areas. A JAX-based implementation further enhances the performance through vectorization of key method components. Extensive experiments, including high-dimensional parameter spaces, uninformative outputs, multiple observations and multimodal posteriors show that our method consistently matches, and often exceeds, the accuracy of state-of-the-art approaches, while reducing the runtime by a substantial margin.

Problem

• Bayesian inference for complex simulators with intractable likelihood functions
• Addressing high computational costs in high-dimensional parameter spaces
• Improving accuracy and efficiency for simulation-based posterior inference

Innovation

• Reformulates simulation as deterministic optimization problems
• Uses gradient-based methods to navigate posterior regions
• Enhances performance with JAX-based vectorized implementation