JEPADepth: Masked Predictive Representation Learning for Self-Supervised Monocular Depth Estimation

2026-07-29

AI Summary

This work addresses the limitations of self-supervised monocular depth estimation, which typically relies on photometric reconstruction loss that entangles depth, pose, and appearance assumptions, thereby constraining representation generalization. For the first time, we introduce the Joint Embedding Predictive Architecture (JEPA) to this task, leveraging a DINOv3-pretrained ViT encoder to predict embeddings of masked target regions from contextual cues in the representation space via a structured masking strategy. The model is jointly optimized with photometric loss and incurs no additional computational overhead at inference. Our approach significantly outperforms baseline methods on KITTI and achieves state-of-the-art or near state-of-the-art zero-shot transfer performance on Make3D and Cityscapes, surpassing leading CNN-based approaches and matching advanced Transformer-based solutions.

Abstract

Self-supervised monocular depth estimation typically relies on photometric reconstruction losses that couple depth, pose, and appearance assumptions. In this paper, we propose JEPADepth, a self-supervised monocular depth framework that incorporates a complementary training objective inspired by Image Joint-Embedding Predictive Architectures (I-JEPA) for self-supervised depth learning. Our method augments a standard photometric pipeline with a masked prediction loss computed in the representation space of a pretrained DINOv3 Vision Transformer encoder. A predictor infers target-region embeddings from visible context-region embeddings under structured masking, and is discarded along with the target encoder at inference time, adding no deployment cost. On KITTI, adding the JEPA objective consistently improves performance over the same DINOv3-based photometric baseline, without changing the inference-time architecture. Compared to prior monocular self-supervised methods, JEPADepth is competitive with state-of-the-art transformer-based approaches and outperforms strong CNN-based baselines on the standard benchmark. In zero-shot transfer (trained on KITTI and evaluated without fine-tuning), JEPADepth achieves the best or near-best performance among the compared methods on both Make3D and Cityscapes across multiple metrics.

Problem

• self-supervised
• monocular depth estimation
• photometric reconstruction
• representation learning
• masked prediction

Innovation

• masked predictive learning
• self-supervised depth estimation
• joint-embedding predictive architecture
• Vision Transformer
• zero-shot transfer