CA-W3D: Leveraging Context-Aware Knowledge for Weakly Supervised Monocular 3D Detection

2025-03-06

AI Summary

Weakly supervised monocular 3D detection suffers from insufficient global context modeling, hindering robust depth and spatial reasoning. To address this, we propose a two-stage context-aware framework. In the first stage, we introduce Region-level Object Contrastive Matching (ROCM), aligning a trainable monocular 3D encoder with a frozen open-vocabulary visual grounding model to inject rich semantic priors. In the second stage, we employ Dual-to-One Distillation (D2OD) for pseudo-label training, jointly optimizing context awareness and geometric accuracy. This work is the first to systematically integrate open-vocabulary visual grounding knowledge into weakly supervised monocular 3D detection. Evaluated on the KITTI benchmark, our method surpasses all state-of-the-art approaches across all major metrics—BEV, 3D, and orientation AP—demonstrating the critical role of context awareness in enhancing weakly supervised 3D detection performance.

Technology Category

• Computer Vision: 3D Computer Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Unsupervised & Self-Supervised Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services

Abstract

Weakly supervised monocular 3D detection, while less annotation-intensive, often struggles to capture the global context required for reliable 3D reasoning. Conventional label-efficient methods focus on object-centric features, neglecting contextual semantic relationships that are critical in complex scenes. In this work, we propose a Context-Aware Weak Supervision for Monocular 3D object detection, namely CA-W3D, to address this limitation in a two-stage training paradigm. Specifically, we first introduce a pre-training stage employing Region-wise Object Contrastive Matching (ROCM), which aligns regional object embeddings derived from a trainable monocular 3D encoder and a frozen open-vocabulary 2D visual grounding model. This alignment encourages the monocular encoder to discriminate scene-specific attributes and acquire richer contextual knowledge. In the second stage, we incorporate a pseudo-label training process with a Dual-to-One Distillation (D2OD) mechanism, which effectively transfers contextual priors into the monocular encoder while preserving spatial fidelity and maintaining computational efficiency during inference. Extensive experiments conducted on the public KITTI benchmark demonstrate the effectiveness of our approach, surpassing the SoTA method over all metrics, highlighting the importance of contextual-aware knowledge in weakly-supervised monocular 3D detection.

Problem

• Addresses weakly supervised monocular 3D detection limitations
• Enhances global context understanding in complex scenes
• Improves 3D reasoning with contextual-aware knowledge

Innovation

• Region-wise Object Contrastive Matching for pre-training
• Dual-to-One Distillation mechanism for pseudo-label training
• Context-Aware Weak Supervision for 3D detection