When World Models Lie: Adaptive Safety Analysis Under Wrong Imaginations

2026-09-28

AI Summary

This study addresses the issue that prediction errors in world models can lead to overconfident estimates in latent-space safety filters. To mitigate this, we propose an adaptive latent safety filter that constructs online uncertainty sets directly from discrepancies between predicted and observed states, eliminating the need for auxiliary signals. By integrating adaptive conformal inference to calibrate the reasoning process, the approach maintains minimal conservatism when the model is accurate while enhancing caution under mismatch conditions. Furthermore, finite-time coverage guarantees are established through latent-space safety value function optimization combined with a pessimistic evaluation strategy. Experimental results demonstrate that the proposed method significantly reduces system failure rates while preserving task completion rates, outperforming existing state-of-the-art approaches.

Abstract

World models offer a powerful substrate for safety reasoning in high-dimensional robotic systems, but they are also fallible: their predictions can be biased, miscalibrated, or confidently wrong. This creates a central challenge for latent-space safety filters, which often learn Hamilton-Jacobi safety value functions on the dynamics of a world model. If the world model is incorrect, the resulting value function can inherit its errors and produce overconfident safety estimates. Existing latent safety filters often rely on auxiliary signals such as ensemble disagreement or value-target consistency residuals for adaptation, but these signals can remain small even when the world model's predictions deviate from observations. We propose an adaptive latent safety filter that calibrates safety reasoning using directly observed world-model error. Our method uses Adaptive Conformal Inference to construct online uncertainty sets from discrepancies between predicted and observation-inferred latent states, then evaluates safety pessimistically by minimizing the learned value function over these sets. This allows the filter to remain minimally conservative when the world model is accurate, while becoming more cautious when observations reveal model mismatch. We provide a finite-time coverage guarantee for the adaptive uncertainty radius. Through simulation and hardware experiments, we show that our method significantly reduces failures relative to state-of-the-art latent safety filters while preserving task completion.

Problem

• World Models
• Latent Safety Filters
• Safety Reasoning
• Model Mismatch
• Robotic Systems

Innovation

• World Models
• Latent Safety Filter
• Adaptive Conformal Inference
• Hamilton-Jacobi Safety Value Function
• Uncertainty Sets