NSF-HRPT: Neural Semantic Field meets Hierarchical Risk Perception Tree for Safety-Critical Scenario Assessment

2026-08-05

AI Summary

This work addresses the challenge of quantifying multi-agent risk in safety-critical scenarios under monocular vision by proposing a unified framework that integrates Neural Semantic Fields (NSF) with a Hierarchical Risk Perception Tree (HRPT). The NSF jointly models scene semantics, trajectory prediction, and the probabilistic distribution of time-to-collision (TTC), while the HRPT enables efficient, parallelized spatial risk reasoning. The approach innovatively incorporates a Sim2Real augmentation strategy that requires no retraining, along with priors from foundation models, substantially enhancing generalization to real-world environments. Experimental results demonstrate state-of-the-art performance on synthetic benchmarks and near-optimal accuracy in TTC estimation and risk localization on real-world datasets, enabling real-time monocular risk perception.

Abstract

The ability to accurately assess and anticipate risks in safety-critical scenarios is crucial for autonomous driving systems. While existing research has made progress in collision prediction, accurately quantifying risk levels from monocular vision inputs remains challenging due to the complex dynamics of multi-agent interactions and the inherent uncertainty in real-world environments. To address these challenges, we present NSF-HRPT, a novel framework that combines learning-based perception with structured reasoning for quantitative risk assessment. Our approach features a Neural Semantic Field (NSF) that learns to model scene semantics, trajectory predictions, and probabilistic Time-to-Collision (TTC) distributions from simulation data. During inference, the pre-trained NSF serves as a prior for our Hierarchical Risk Perception Tree (HRPT), which enables efficient parallel computation and spatial reasoning about multi-agent risks. Additionally, we introduce a Sim2Real enhancement strategy that improves real-world applicability without retraining by incorporating priors from foundation models. Extensive evaluations demonstrate that our framework achieves state-of-the-art performance on synthetic benchmarks and delivers competitive, near-state-of-the-art results on real-world datasets for both TTC estimation accuracy and risk localization precision. The proposed method provides an effective solution for real-time risk awareness from monocular camera inputs.

Problem

• risk assessment
• monocular vision
• multi-agent interaction
• safety-critical scenarios
• uncertainty

Innovation

• Neural Semantic Field
• Hierarchical Risk Perception Tree
• Time-to-Collision estimation
• Sim2Real transfer
• Monocular risk assessment