Distilling Privileged Control Barrier Functions into RGB-Only Safety Filters for Dynamic Visual Navigation

2026-09-28

AI Summary

This study addresses the collision-prone nature of end-to-end RGB visual navigation in dynamic environments and the impracticality of existing methods that rely on real-time rendering or explicit reconstruction for onboard deployment. To this end, this work proposes a teacher-student distillation framework that transfers the safety policy of a Control Barrier Function (CBF) teacher with privileged information to a student filter relying solely on RGB inputs, enabling real-time safe control without 3D reconstruction. The core innovation lies in formulating constraints based on visible obstacles while introducing velocity uncertainty modeling and action augmentation techniques to effectively bridge the information gap between teacher and student. Experimental results demonstrate that the proposed method significantly outperforms existing visual CBF baselines, substantially enhancing the safety and robustness of RGB-based navigation policies in dynamic scenarios.

Abstract

RGB-only end-to-end visual navigation policies remain vulnerable to collisions in real-world dynamic environments, motivating a dedicated safety layer. Existing visual Control Barrier Function (CBF) approaches seek to provide safety from RGB observations, but often rely on real-time rendering or explicit scene reconstruction and are primarily designed for static scenes, limiting their practicality for onboard deployment. We propose a teacher-student visual distillation framework that transfers the safety behavior of a privileged CBF teacher to an RGB-only student filter for dynamic environments. The student maps a short RGB history, robot velocity, and a nominal control action directly to a safe action, while the teacher uses ground-truth robot and obstacle states in a real-to-sim dynamic Gaussian Splatting environment. To reduce the teacher-student information gap, the teacher constructs safety constraints only from obstacles observable within the student's RGB history. It also accounts for obstacle-velocity uncertainty to improve robustness to motion variations, while action augmentation exposes the student to diverse safe and unsafe nominal actions to better capture the safety boundary. At deployment, the student requires only RGB observations and robot velocity, without explicit 3D reconstruction or online rendering. Experiments show that the proposed method outperforms visual CBF baselines and improves the safety of RGB-based navigation policies under dynamic obstacle motion. Project page: https://syeon-yoo.github.io/distill-cbf-site/.

Problem

• Visual Navigation
• Control Barrier Functions
• Dynamic Environments
• Safety Filter
• RGB-Only

Innovation

• Control Barrier Function
• Knowledge Distillation
• Visual Navigation
• Dynamic Gaussian Splatting
• Safety Filter