RadarDistill: Boosting Radar-Based Object Detection Performance via Knowledge Distillation from LiDAR Features

2024-03-08

Computer Vision and Pattern Recognition

AI Summary

Sparse and noisy radar data severely degrade representation capability for 3D object detection. To address this, we propose RadarDistill, the first knowledge distillation framework enabling efficient, selective, and structure-aware cross-modal knowledge transfer from LiDAR to radar. Our method comprises three synergistic modules: (1) Cross-Modal Alignment (CMA), which matches feature density across modalities via multi-layer dilated convolutions; (2) Activation-Driven Feature Distillation (AFD), which performs region-selective knowledge transfer based on activation magnitude thresholds; and (3) Proposal-Driven Feature Distillation (PFD), which enforces local structural imitation under ground-truth proposal constraints. Evaluated on nuScenes, our radar-only model achieves state-of-the-art performance—20.5% mAP and 43.7% NDS—while also substantially improving camera-radar fusion accuracy.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

The inherent noisy and sparse characteristics of radar data pose challenges in finding effective representations for 3D object detection. In this paper, we propose RadarD-istill, a novel knowledge distillation (KD) method, which can improve the representation of radar data by leveraging LiDAR data. RadarDistill successfully transfers de-sirable characteristics of LiDAR features into radar features using three key components: Cross-Modality Alignment (CMA), Activation-based Feature Distillation (AFD), and Proposal-based Feature Distillation (PFD). CMA en-hances the density of radar features by employing multiple layers of dilation operations, effectively addressing the challenge of inefficient knowledge transfer from LiDAR to radar. AFD selectively transfers knowledge based on regions of the LiDAR features, with a specific focus on areas where activation intensity exceeds a predefined thresh-old. P FD similarly guides the radar network to selectively mimic features from the LiDAR network within the object proposals. Our comparative analyses conducted on the nuScenes datasets demonstrate that RadarDistill achieves state-of-the-art (SOTA) performance for radar-only object detection task, recording 20.5% in mAP and 43.7% in NDS. Also, RadarDistill significantly improves the performance of the camera-radar fusion model.

Problem

• Improving radar-based 3D object detection performance
• Addressing noisy and sparse radar data characteristics
• Transferring LiDAR features to enhance radar representations

Innovation

• Knowledge distillation from LiDAR to radar
• Cross-Modality Alignment with dilation operations
• Activation-based and proposal-based feature distillation