RadarNeXt: Real-Time and Reliable 3D Object Detector Based On 4D mmWave Imaging Radar

2025-01-04

AI Summary

To address the challenge of balancing accuracy, inference speed, and embedded deployment for 4D mmWave radar-based 3D object detection in autonomous driving, this paper proposes a lightweight and efficient detection framework. We design a reparameterizable neural network architecture integrated with a Multi-path Deformable Foreground Enhancement Network (MDFEN), which leverages deformable attention and multi-path feature fusion to effectively suppress background clutter and model irregular foreground structures. Evaluated on View-of-Delft and TJ4DRadSet, our method achieves 50.48% and 32.30% mAP, respectively. It attains real-time inference speeds of 67.10 FPS on an RTX A4000 and 28.40 FPS on a Jetson AGX Orin, while maintaining low parameter count and computational overhead. The framework thus achieves a superior trade-off among detection accuracy, latency, and model efficiency, enabling robust edge deployment for real-time autonomous driving applications.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

3D object detection is crucial for Autonomous Driving (AD) and Advanced Driver Assistance Systems (ADAS). However, most 3D detectors prioritize detection accuracy, often overlooking network inference speed in practical applications. In this paper, we propose RadarNeXt, a real-time and reliable 3D object detector based on the 4D mmWave radar point clouds. It leverages the re-parameterizable neural networks to catch multi-scale features, reduce memory cost and accelerate the inference. Moreover, to highlight the irregular foreground features of radar point clouds and suppress background clutter, we propose a Multi-path Deformable Foreground Enhancement Network (MDFEN), ensuring detection accuracy while minimizing the sacrifice of speed and excessive number of parameters. Experimental results on View-of-Delft and TJ4DRadSet datasets validate the exceptional performance and efficiency of RadarNeXt, achieving 50.48 and 32.30 mAPs with the variant using our proposed MDFEN. Notably, our RadarNeXt variants achieve inference speeds of over 67.10 FPS on the RTX A4000 GPU and 28.40 FPS on the Jetson AGX Orin. This research demonstrates that RadarNeXt brings a novel and effective paradigm for 3D perception based on 4D mmWave radar.

Problem

• 3D object detection
• real-time processing
• autonomous driving

Innovation

• 4D mmWave Radar
• MDFEN Network
• Real-time 3D Object Detection