Learning Monocular Depth from Events via Egomotion Compensation

2024-12-26

AI Summary

To address insufficient physical modeling, underutilization of temporal information, and parameter redundancy caused by black-box designs in event-camera-based monocular depth estimation, this paper proposes an interpretable depth estimation framework grounded in motion-compensated physical priors. Our method integrates event-stream processing, focus assessment, cost-volume prediction, and multi-scale consistency constraints, enabling probabilistic depth inference. Key contributions include: (1) the first motion-compensation-driven likelihood modeling of depth hypotheses; (2) a Focus Cost Discrimination (FCD) module that guides cost-volume discrimination via focus sharpness; and (3) an Inter-Hypotheses Cost Aggregation (IHCA) module that fuses multiple depth hypotheses to enhance noise robustness. Evaluated on both real and synthetic datasets, our approach achieves up to a 10% reduction in AbsRel error over state-of-the-art methods, significantly improving accuracy and generalization under high-speed motion and low-illumination conditions.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Event cameras are neuromorphically inspired sensors that sparsely and asynchronously report brightness changes. Their unique characteristics of high temporal resolution, high dynamic range, and low power consumption make them well-suited for addressing challenges in monocular depth estimation (e.g., high-speed or low-lighting conditions). However, current existing methods primarily treat event streams as black-box learning systems without incorporating prior physical principles, thus becoming over-parameterized and failing to fully exploit the rich temporal information inherent in event camera data. To address this limitation, we incorporate physical motion principles to propose an interpretable monocular depth estimation framework, where the likelihood of various depth hypotheses is explicitly determined by the effect of motion compensation. To achieve this, we propose a Focus Cost Discrimination (FCD) module that measures the clarity of edges as an essential indicator of focus level and integrates spatial surroundings to facilitate cost estimation. Furthermore, we analyze the noise patterns within our framework and improve it with the newly introduced Inter-Hypotheses Cost Aggregation (IHCA) module, where the cost volume is refined through cost trend prediction and multi-scale cost consistency constraints. Extensive experiments on real-world and synthetic datasets demonstrate that our proposed framework outperforms cutting-edge methods by up to 10% in terms of the absolute relative error metric, revealing superior performance in predicting accuracy.

Problem

• Event Camera
• Depth Estimation
• Efficiency

Innovation

• Depth Estimation
• Event Camera
• Explainable Framework