DroneFL: Federated Learning for Multi-UAV Visual Target Tracking

2025-09-25

AI Summary

Multi-UAV visual target tracking faces challenges including limited onboard computational resources, strong data heterogeneity across devices, and tight coupling between trajectory prediction and collaborative planning. Method: We propose a lightweight federated learning framework featuring a local model that freezes the YOLO backbone and employs a shallow Transformer; we introduce a position-invariant architecture and altitude-based adaptive normalization to mitigate data heterogeneity, and design a cloud-side multi-UAV trajectory fusion mechanism to tightly couple trajectory optimization with distributed training. Results: Our approach achieves real-time performance (<30 ms/frame) on Raspberry Pi 5, with an average upload bandwidth of only 1.56 KBps. Compared to non-federated baselines, it reduces prediction error by 6%–83% and shortens average tracking distance by 0.4%–4.6%, significantly improving both resource efficiency and collaborative tracking accuracy.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multi-robot target tracking is a fundamental problem that requires coordinated monitoring of dynamic entities in applications such as precision agriculture, environmental monitoring, disaster response, and security surveillance. While Federated Learning (FL) has the potential to enhance learning across multiple robots without centralized data aggregation, its use in multi-Unmanned Aerial Vehicle (UAV) target tracking remains largely underexplored. Key challenges include limited onboard computational resources, significant data heterogeneity in FL due to varying targets and the fields of view, and the need for tight coupling between trajectory prediction and multi-robot planning. In this paper, we introduce DroneFL, the first federated learning framework specifically designed for efficient multi-UAV target tracking. We design a lightweight local model to predict target trajectories from sensor inputs, using a frozen YOLO backbone and a shallow transformer for efficient onboard training. The updated models are periodically aggregated in the cloud for global knowledge sharing. To alleviate the data heterogeneity that hinders FL convergence, DroneFL introduces a position-invariant model architecture with altitude-based adaptive instance normalization. Finally, we fuse predictions from multiple UAVs in the cloud and generate optimal trajectories that balance target prediction accuracy and overall tracking performance. Our results show that DroneFL reduces prediction error by 6%-83% and tracking distance by 0.4%-4.6% compared to a distributed non-FL framework. In terms of efficiency, DroneFL runs in real time on a Raspberry Pi 5 and has on average just 1.56 KBps data rate to the cloud.

Problem

• Enabling federated learning for multi-UAV visual target tracking
• Addressing data heterogeneity in federated learning across UAVs
• Balancing trajectory prediction accuracy with tracking performance optimization

Innovation

• Lightweight local model with frozen YOLO backbone
• Position-invariant architecture using adaptive instance normalization
• Cloud-based fusion for multi-UAV trajectory optimization