Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation

Unknown Date

AI Summary

This work addresses the scarcity of ground-truth labels in 4D radar scene flow estimation and the reliance on expensive LiDAR sensors or inefficient self-supervision by proposing a weakly supervised, iterative framework that leverages only monocular images and odometry. The method employs off-the-shelf 2D visual tracking and segmentation to generate instance masks, which are back-projected into 3D space to provide semantic guidance. Concurrently, it fuses odometry with radar motion cues to formulate a static rigid constraint loss, thereby circumventing the need for complex multi-task architectures or LiDAR supervision. Evaluated on the View-of-Delft dataset, the approach surpasses existing fully supervised and cross-modal LiDAR-supervised methods, achieving state-of-the-art performance.

Abstract

Due to the difficulty of obtaining ground-truth data for 4D radar scene flow estimation, previous methods typically rely on either self-supervised losses or cross-modal supervision using 3D LiDAR data, 2D images, and odometry. However, self-supervised approaches often yield suboptimal results due to radar's inherently low-fidelity measurements, while existing cross-modal supervised methods introduce complex multi-task architecture and require costly LiDAR sensors to generate pseudo radar scene flow labels from pretrained 3D tracking models. To overcome these limitations, we propose a task-specific iterative framework for weakly supervised radar scene flow learning, using only images and odometry for auxiliary supervision during training. Specially, we establish two novel instance-aware self-supervised losses by exploiting off-the-shelf 2D tracking and segmentation algorithms to obtain tracked instance masks, which are back-projected into 3D space to provide instance-level semantic guidance; for static regions, we integrate vehicle odometry with radar's intrinsic motion cues to construct a rigid static loss. Extensive experiments on the real-world View-of-Delft (VoD) dataset demonstrate that our method not only surpasses state-of-the-art cross-modal supervised approaches that rely on 3D multi-object tracking on dense LiDAR point clouds but also outperforms existing fully supervised scene flow estimation methods. The code is open-sourced at \href{https://github.com/FuJingyun/IterFlow}{https://github.com/FuJingyun/IterFlow}.

Problem

• 4D radar
• scene flow estimation
• weakly supervised learning
• cross-modal supervision
• ground-truth scarcity

Innovation

• weakly supervised learning
• 4D radar
• scene flow estimation
• cross-modal supervision
• instance-aware self-supervision