Learned Non-Maximum Suppression for 3D Object Detection

2026-06-02

AI Summary

This work addresses the limitations of conventional heuristic non-maximum suppression (NMS) in LiDAR-based 3D object detection, which struggles to handle densely overlapping bounding boxes, compromising detection compactness and reliability. To overcome this, the authors introduce learnable relational modeling into 3D NMS for the first time, proposing two end-to-end trainable post-processing modules: D2D-Rescore, which leverages detection-to-detection attention, and GossipNet3D, based on local message passing in bird’s-eye-view space. A metric-aware matching strategy aligned with the nuScenes evaluation protocol is also devised. Without modifying the backbone network, the method achieves significant improvements in mAP, NDS, and true positive quality on nuScenes—particularly for small objects and rare classes—while incurring minimal computational overhead.

Abstract

Post-processing is a critical stage in LiDAR-based 3D object detection, where dense and overlapping proposals must be filtered for compact and reliable perception. This work introduces two learned filtering modules that replace heuristic non-maximum suppression (NMS) by leveraging relations among detections. D2D-Rescore employs transformer-based detection-to-detection (D2D) attention, while GossipNet3D adapts the 2D GossipNet concept to 3D through localized message passing in bird's-eye view. A metric-aware matching strategy aligned with the nuScenes evaluation protocol ensures consistent training and validation behavior, improving overall detection performance. Both approaches improve mean average precision (mAP), nuScenes detection score (NDS), and true positive quality compared to CircleNMS, particularly for small and infrequent classes, while adding minimal computational overhead. These results demonstrate that learned, detection-level filtering can enhance 3D detector reliability without modifying the base network, offering a principled alternative to heuristic suppression. Code is available at https://github.com/rst-tu-dortmund/learned-3d-nms .

Problem

• 3D object detection
• non-maximum suppression
• LiDAR
• post-processing
• detection filtering

Innovation

• learned NMS
• 3D object detection
• detection-to-detection attention
• GossipNet3D
• metric-aware matching