Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions

2026-07-30

AI Summary

Monocular depth estimation is prone to visual ambiguities on non-Lambertian surfaces and under adverse weather conditions. To address this challenge, this work proposes CapDepth, a novel framework that, for the first time, incorporates spatially rich, detailed long-form textual descriptions into the depth estimation task. The approach employs a structured caption template, a dynamic caption encoder, a progressive masked attention mechanism, and a text-adaptive decoder featuring stable adaptive layer normalization to enable fine-grained language guidance. Extensive experiments demonstrate that CapDepth reduces depth errors by 25.0% on non-Lambertian surfaces and by 22.0% in adverse weather scenarios, significantly outperforming current state-of-the-art methods.

Abstract

Monocular depth estimation (MDE) faces challenges with non-Lambertian surfaces and adverse weather conditions due to the visual ambiguities inherent in single-image limited information. Existing works address them in isolation via image inpainting or augmentation, yielding limited robustness gains. Language, as a powerful complementary modality to vision, is demonstrated to enhance the visual perception capabilities of vision-language models (VLMs) via detailed long captions. However, prior language-integrated MDE methods fail to fully harness this potential due to short text input with limited information, coarse global text feature learning, and limited language guidance during depth decoding. To address these limitations, we propose CapDepth, a novel framework for robust MDE that leverages guidance from detailed long captions to alleviate visual ambiguities in both challenging scenarios. First, we design a detailed long caption input template that explicitly conveys rich spatial relationships among multiple atom sentences. Second, a dynamic caption encoder is introduced to extract fine-grained depth-relevant text features via progressive masked attention. Finally, we propose a text-adaptive decoder that guides enhanced depth decoding with text features via stable adaptive layer normalization. Extensive experiments validate the efficacy of CapDepth, which outperforms state-of-the-art methods, achieving depth error reductions of 25.0% on non-Lambertian surfaces and 22.0% under adverse weather conditions.

Problem

• monocular depth estimation
• visual ambiguity
• non-Lambertian surfaces
• adverse weather conditions

Innovation

• monocular depth estimation
• vision-language models
• detailed long captions
• adaptive layer normalization
• visual ambiguity