How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy

2025-08-12

AI Summary

Visual-control autonomous robots face challenges in reliably predicting long-horizon safety under partial observability and distributional shift. Method: We propose a calibration-aware safety prediction framework that requires no explicit state model. It employs a world model combining a variational autoencoder with a recurrent predictor to forecast latent-space trajectories instead of high-dimensional raw observations. A dual-path architecture—supporting both monolithic and compositional prediction—is introduced, augmented with unsupervised domain adaptation to mitigate distributional shift. Safety probabilities are calibrated via conformal prediction to provide statistically valid confidence guarantees. Results: Experiments on three benchmarks demonstrate substantial reductions in false-positive rates. Compositional prediction consistently outperforms monolithic variants, and calibrated risk assessments maintain high accuracy and reliable confidence intervals even over extended time horizons.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness
• Intelligent Robots: State Estimation

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

Autonomous robots that rely on deep neural network controllers pose critical challenges for safety prediction, especially under partial observability and distribution shift. Traditional model-based verification techniques are limited in scalability and require access to low-dimensional state models, while model-free methods often lack reliability guarantees. This paper addresses these limitations by introducing a framework for calibrated safety prediction in end-to-end vision-controlled systems, where neither the state-transition model nor the observation model is accessible. Building on the foundation of world models, we leverage variational autoencoders and recurrent predictors to forecast future latent trajectories from raw image sequences and estimate the probability of satisfying safety properties. We distinguish between monolithic and composite prediction pipelines and introduce a calibration mechanism to quantify prediction confidence. In long-horizon predictions from high-dimensional observations, the forecasted inputs to the safety evaluator can deviate significantly from the training distribution due to compounding prediction errors and changing environmental conditions, leading to miscalibrated risk estimates. To address this, we incorporate unsupervised domain adaptation to ensure robustness of safety evaluation under distribution shift in predictions without requiring manual labels. Our formulation provides theoretical calibration guarantees and supports practical evaluation across long prediction horizons. Experimental results on three benchmarks show that our UDA-equipped evaluators maintain high accuracy and substantially lower false positive rates under distribution shift. Similarly, world model-based composite predictors outperform their monolithic counterparts on long-horizon tasks, and our conformal calibration provides reliable statistical bounds.

Problem

• Predicting safety in vision-controlled autonomous robots under uncertainty
• Ensuring calibrated safety forecasts without state or observation models
• Addressing distribution shifts in long-horizon safety predictions

Innovation

• Uses variational autoencoders for latent trajectory forecasting
• Incorporates unsupervised domain adaptation for robustness
• Provides conformal calibration for reliable statistical bounds