NCGR: Noise-Conditional Gated Rectification for Camera Extrinsic Perturbations in BEV 3D Object Detection

2026-08-04

AI Summary

This work addresses the performance degradation in BEV-based 3D object detection caused by sampling bias in spatial cross-attention due to camera extrinsic perturbations. To mitigate this without explicitly estimating full six-degree-of-freedom extrinsics, the authors propose a noise-conditioned gated correction mechanism that lightweightly predicts 2D correction offsets for each query–camera pair and rectifies projected sampling locations via camera-level gating prior to deformable attention. Integrated within a teacher–student framework and trained with BEV consistency supervision, the method enables blind inference without requiring perturbation metadata. Evaluated on nuScenes with dynamic extrinsic perturbations across five cameras, the approach achieves an NDS of 39.69%, substantially outperforming BEVFormer (28.00%) and CAPE (33.23%), while maintaining comparable performance in unperturbed settings.

Abstract

Camera-based bird's-eye-view (BEV) 3D detection typically assumes accurate and fixed camera extrinsics. In detectors using spatial cross-attention (SCA), extrinsic perturbations displace the image-plane projections of BEV reference points, causing queries to sample features from incorrect regions and degrading detection performance. To address this failure mode, Noise-Conditional Gated Rectification (NCGR) is proposed to compensate for projection errors without explicitly estimating a full six-degree-of-freedom extrinsic correction. For each query-camera pair, a 2D rectification offset is predicted and modulated by a camera-level gate to rectify the base projection before native deformable sampling. During training, the perturbation-derived quantities used to construct the condition and gate are gradually replaced through scheduled interpolation by counterparts generated from an auxiliary scalar predicted from camera features. This transition enables blind inference without perturbation metadata. During training, a weight-shared clean-teacher/perturbed-student pair is used, and the rectification module is supervised by a BEV-consistency objective between the two branches. NCGR is evaluated on nuScenes with simulated dynamic and static extrinsic perturbations. In a five-camera dynamic stress test, NCGR achieves 39.69% NDS, compared with 28.00% for BEVFormer and 33.23% for CAPE. Under clean extrinsics, NCGR maintains performance comparable to that of BEVFormer.

Problem

• camera extrinsic perturbations
• BEV 3D object detection
• spatial cross-attention
• projection errors
• detection performance degradation

Innovation

• Noise-Conditional Gated Rectification
• Camera Extrinsic Perturbations
• BEV 3D Object Detection
• Spatial Cross-Attention
• Deformable Sampling