Two-stage Design for Failure Probability Estimation with Gaussian Process Surrogates

2024-10-06

AI Summary

This work addresses the challenge of estimating small failure probabilities under stochastic inputs in computationally expensive deterministic simulations. We propose a two-stage adaptive budget allocation framework: in Stage I, a Gaussian process surrogate is sequentially trained using a contour-localization strategy; in Stage II, remaining simulation budget is greedily allocated to critical regions—guided by classification entropy—to perform high-fidelity evaluations. A hybrid Monte Carlo estimator is then constructed by integrating surrogate predictions with observed high-fidelity responses. Our method introduces the first “exploration–exploitation decoupled” budget allocation paradigm, overcoming reliability limitations inherent in pure surrogate-based Monte Carlo and importance sampling. Experiments across multiple benchmark functions and an airfoil flow simulation demonstrate that the approach achieves significantly improved accuracy and robustness using only several hundred high-fidelity evaluations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

We tackle the problem of quantifying failure probabilities for expensive deterministic computer experiments with stochastic inputs. The computational cost of the computer simulation prohibits direct Monte Carlo (MC) and necessitates a surrogate model, turning the problem into a two-stage enterprise (surrogate training followed by probability estimation). Limited budgets create a design problem: how should expensive evaluations be allocated between and within the training and estimation stages? One may use the entire evaluation budget to sequentially train the surrogate through contour location (CL), with failure probabilities then estimated solely from the surrogate (we call it"surrogate MC"). But extended CL offers diminishing returns, and surrogate MC relies too stringently on surrogate accuracy. Alternatively, a partially trained surrogate may inform importance sampling, but this can provide erroneous results when budgets are limited. Instead we propose a two-stage design: starting with sequential CL, halting CL once learning has plateaued, then greedily allocating the remaining budget to MC samples with high classification entropy. Ultimately, we employ a"hybrid MC"estimator which leverages the trained surrogate in conjunction with the true responses observed in this second stage. Our unique two-stage design strikes an appropriate balance between exploring and exploiting, and outperforms alternatives, including both of the aforementioned approaches, on a variety of benchmark exercises. With these tools, we are able to effectively estimate small failure probabilities with only hundreds of simulator evaluations, showcasing functionality with both shallow and deep Gaussian process surrogates, and deploying our method on a simulation of fluid flow around an airfoil.

Problem

• Estimating failure probabilities with limited computational budget
• Optimizing surrogate model training for accurate classification
• Improving efficiency over existing sequential contour location methods

Innovation

• Two-stage surrogate design for failure probability estimation
• Stage 1: Sequential contour location for failure contour
• Stage 2: Exploitative MC sampling with highest entropy