Enhanced Importance Sampling through Latent Space Exploration in Normalizing Flows

2025-01-06

AI Summary

Importance sampling in rare-event simulation suffers from low efficiency when the proposal distribution poorly matches the target distribution. To address this, we propose a novel importance sampling framework based on latent-space optimization of normalizing flows. Our approach innovatively shifts proposal distribution updates to the interpretable, differentiable latent space of normalizing flows—leveraging their invertible transformations to guarantee valid sampling in the target space—while integrating latent-space gradient optimization with weighted importance sampling for efficient biasing. Evaluated on high-risk scenarios including autonomous racing and aircraft ground collision avoidance, our method reduces estimation variance by 3.2× and decreases rare-event estimation error by over 40% compared to baseline approaches. The framework significantly improves estimation accuracy, robustness, and sampling efficiency, offering a principled solution to proposal mismatch in safety-critical simulations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Importance sampling is a rare event simulation technique used in Monte Carlo simulations to bias the sampling distribution towards the rare event of interest. By assigning appropriate weights to sampled points, importance sampling allows for more efficient estimation of rare events or tails of distributions. However, importance sampling can fail when the proposal distribution does not effectively cover the target distribution. In this work, we propose a method for more efficient sampling by updating the proposal distribution in the latent space of a normalizing flow. Normalizing flows learn an invertible mapping from a target distribution to a simpler latent distribution. The latent space can be more easily explored during the search for a proposal distribution, and samples from the proposal distribution are recovered in the space of the target distribution via the invertible mapping. We empirically validate our methodology on simulated robotics applications such as autonomous racing and aircraft ground collision avoidance.

Problem

• Importance Sampling
• Rare Events
• Monte Carlo Simulation

Innovation

• Normalized Flows
• Importance Sampling
• Rare Event Simulation