DepthART: Scaling Foundation Monocular Depth to Tiny Models

2026-07-19

AI Summary

This work addresses the challenges faced by existing lightweight monocular depth estimation models, which struggle to balance accuracy, generalization, and deployment efficiency due to overfitting from distributional bias and metric instability under varying camera intrinsics. To overcome these limitations, we propose DepthART, a compact edge-oriented model that integrates bias-resistant data sampling, a camera-intrinsic-conditioned fine-tuning mechanism with a frozen distilled encoder, and a scalable architecture design—all without increasing model complexity. The resulting DepthART-S achieves a zero-shot δ₁ accuracy of 0.964 on NYU Depth v2, approaching the performance of much larger models, while enabling real-time inference on edge devices such as the Jetson Nano and reaching up to 347 FPS on an RTX A6000 GPU.

Abstract

Recent geometric foundation models (e.g., Metric3D, Depth Anything and UniDepth) have substantially improved monocular depth estimation (MDE) in both cross-scene generalization and metric-scale prediction, yet these gains have not translated to tiny models. We bridge this gap with DepthART (Depth Anything Rethought for Tiny Models), which is a compact MDE model for on-device deployment across diverse scenes. We first identify two capacity-driven bottlenecks in tiny models: (i) overfitting to dataset-specific distribution bias and (ii) unstable metric adaptation under camera shift, where full fine-tuning easily damages transferable geometry. Accordingly, DepthART combines two simple but effective strategies: a bias-resistant data sampling scheme to reduce distribution bias under the same training budget, and a camera-conditioned fine-tuning protocol that freezes the distilled encoder and adjusts metric scale conditioned on intrinsics while better preserving cross-dataset generalization. Across datasets, DepthART consistently surpasses previous tiny baselines in both zero-shot generalization and metric accuracy (e.g., zero-shot $δ_1$=0.964 for DepthART-S on NYUD v2), and in some cases approaches heavy models. We further provide a scalable model family, with DepthART-S reaching 347/245 FPS (strict FP32) on an RTX A6000 at $224^2/448^2$, 102 FPS (TF32) on a Orin NX 8GB, and over 15 FPS (FP32) on a Jetson Nano 4GB.

Problem

• monocular depth estimation
• tiny models
• cross-scene generalization
• metric-scale prediction
• on-device deployment

Innovation

• monocular depth estimation
• tiny models
• distribution bias
• camera-conditioned fine-tuning
• on-device deployment