Geometry-Aware Fisheye-LiDAR Fusion for Robust 3D Object Detection in Low-Overlap Setups

2026-06-07

AI Summary

This work addresses the challenges of feature distortion and information loss arising from geometric mismatch between dual fisheye cameras and roof-mounted LiDAR under low-overlap, high-distortion configurations. To this end, we propose the Geometry-Aware Hybrid Fusion (GA-HF) framework, which pioneers BEV fusion of fisheye images and LiDAR data. Our approach employs a distortion-aware LSS module to map image features into a polar-coordinate BEV representation, preserving angular density, while processing LiDAR features in Cartesian space to maintain metric accuracy. A dual-attention warping correction module is further introduced to align these heterogeneous features effectively. Evaluated on KITTI-360, Dur360BEV, and Fisheye3DOD benchmarks, GA-HF achieves state-of-the-art performance, notably improving the NDS by 4.2% on KITTI-360 and significantly reducing orientation error.

Abstract

As autonomous systems expand from capital-intensive robotaxis to cost-sensitive logistics, sensor configurations are increasingly optimized for coverage-per-cost. A prevalent sparse-view setup utilizes dual-fisheye cameras with a roof-mounted LiDAR, introducing severe geometric challenges: extreme radial distortion, minimal overlap, and misalignment between spherical projections and rectilinear grids. BEV fusion algorithms typically force image and point cloud modalities into unified Cartesian grids early in the pipeline, causing significant feature distortion and information loss for wide-view fisheye cameras. To address this, we propose a Geometry-Aware Hybrid Fusion (GA-HF) framework that explicitly accounts for fisheye geometry and BEV feature distortion, where fisheye features are lifted into a polar BEV grid via a Distortion-Aware Lift-Splat-Shoot (LSS) module to preserve native angular density, while LiDAR features are processed in native Cartesian space for metric fidelity of bounding box regression. To bridge these heterogeneous streams, we introduce a Dual-Attention Warping Correction module that applies spatial and channel attention to the warped camera features before fusion, explicitly suppressing artifacts in low-quality peripheral regions while enhancing high-quality semantic cues. GA-HF is evaluated on three benchmarks: KITTI-360, Dur360BEV, and Fisheye3DOD datasets. To the best of our knowledge, it is the first approach to explore LiDAR-fisheye camera fusion. On KITTI-360, GA-HF improves NDS by 4.2% over Cartesian baselines; on Dur360BEV, it surpasses both LiDAR-only and BEVFusion, while significantly reducing orientation error despite the geometric distortions; on Fisheye3DOD, it attains the highest detection score among all fusion methods.

Problem

• fisheye-LiDAR fusion
• 3D object detection
• low-overlap setup
• geometric distortion
• BEV fusion

Innovation

• Geometry-Aware Fusion
• Fisheye-LiDAR Fusion
• Polar BEV Representation
• Distortion-Aware LSS
• Dual-Attention Warping