FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity

2026-08-02

AI Summary

This work addresses the performance degradation in depth estimation for heterogeneous robotic platforms under federated learning, caused by significant and ambiguous domain shifts encompassing differences in platform hardware, operating environments, and depth distributions. To tackle this challenge, the authors propose a descriptor-driven soft clustering federated learning framework that relaxes the conventional hard clustering assumption by allowing clients to participate in multiple clusters, thereby modeling continuous and fuzzy domain transitions. The study further introduces, for the first time, two realistic non-IID scenarios specifically tailored to robotic depth estimation. Without sharing raw data, the proposed method consistently outperforms standard federated learning and existing clustered federated approaches across multiple depth estimation models, demonstrating superior generalization, robustness, and practical deployability.

Abstract

Although recent robot perception research emphasizes training on data from diverse environments to improve generalization, most existing methods still rely on centralized learning, which is inefficient and difficult to scale across heterogeneous robot platforms. Federated learning (FL) offers an alternative by enabling distributed training without raw data transfer, but it suffers from severe performance degradation under domain shifts caused by heterogeneity across clients. In real robotic deployments, data distributions often overlap across platforms, environments, and sensing conditions, making it difficult to partition clients into clearly separated domains. However, this characteristic breaks the assumption of clearly separable client domains commonly used in clustered FL. To address this gap in robot perception, particularly in depth estimation, we introduce two realistic and unexplored non-IID scenarios that reflect heterogeneity in terms of platform, environment, and depth distribution. We then propose FeDepth, a descriptor-based clustered FL framework that models client relationships through soft clustering. Unlike hard clustering methods that assume clearly separated clusters, FeDepth allows clients to participate in multiple clusters, capturing continuous and ambiguous domain transitions commonly observed in robotic environments. Extensive experiments demonstrate that FeDepth consistently improves robustness over standard FL and clustered FL baselines across multiple depth estimation architectures, providing a practical and effective solution for federated robot perception. Our project page is available at https://vision3d-lab.github.io/fedepth/.

Problem

• federated learning
• depth estimation
• robot heterogeneity
• non-IID
• domain shift

Innovation

• Federated Learning
• Depth Estimation
• Robot Heterogeneity
• Soft Clustering
• Non-IID