LiDAR Prompted Spatio-Temporal Multi-View Stereo for Autonomous Driving

2026-03-04

AI Summary

Existing deep learning–based depth estimation methods for autonomous driving struggle to simultaneously achieve high metric accuracy, multi-view and temporal consistency, and cross-domain generalization. This work proposes DriveMVS, a novel framework that uniquely integrates sparse yet precise LiDAR measurements as both hard geometric priors and soft feature guidance within a multi-view stereo matching network. Furthermore, a spatiotemporal consistency decoder is introduced to fuse cues from multiple views, temporal sequences, and LiDAR observations into an optimized cost volume. Extensive experiments demonstrate that DriveMVS significantly outperforms state-of-the-art methods across multiple benchmarks, achieving leading performance in metric accuracy, inter-frame stability, and zero-shot cross-domain transferability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Large-scale security measurements

Abstract

Accurate metric depth is critical for autonomous driving perception and simulation, yet current approaches struggle to achieve high metric accuracy, multi-view and temporal consistency, and cross-domain generalization. To address these challenges, we present DriveMVS, a novel multi-view stereo framework that reconciles these competing objectives through two key insights: (1) Sparse but metrically accurate LiDAR observations can serve as geometric prompts to anchor depth estimation in absolute scale, and (2) deep fusion of diverse cues is essential for resolving ambiguities and enhancing robustness, while a spatio-temporal decoder ensures consistency across frames. Built upon these principles, DriveMVS embeds the LiDAR prompt in two ways: as a hard geometric prior that anchors the cost volume, and as soft feature-wise guidance fused by a triple-cue combiner. Regarding temporal consistency, DriveMVS employs a spatio-temporal decoder that jointly leverages geometric cues from the MVS cost volume and temporal context from neighboring frames. Experiments show that DriveMVS achieves state-of-the-art performance on multiple benchmarks, excelling in metric accuracy, temporal stability, and zero-shot cross-domain transfer, demonstrating its practical value for scalable, reliable autonomous driving systems.

Problem

• metric depth
• multi-view consistency
• temporal consistency
• cross-domain generalization
• autonomous driving

Innovation

• LiDAR prompting
• spatio-temporal consistency
• multi-view stereo
• metric depth estimation
• cross-domain generalization