Federated Learning With Energy Harvesting Devices: An MDP Framework

2024-05-17

arXiv.org

AI Summary

To address rapid battery depletion in energy-constrained edge devices during federated learning (FL), caused by local training and communication overheads and degrading convergence performance, this paper proposes an energy-harvesting-aware joint optimization framework. First, we derive a novel FL convergence bound incorporating dynamic energy constraints, explicitly characterizing how energy availability governs convergence rate. Second, we formulate joint device scheduling and power control as a Markov decision process (MDP) and theoretically establish the monotonicity of the optimal policy with respect to battery level and channel state. Leveraging this structural insight, we design a low-complexity structure-guided algorithm and a monotonicity-enhanced deep reinforcement learning method. Theoretical analysis proves the asymptotic optimality of our algorithms. Experiments on real-world datasets demonstrate over 25% faster convergence, a 32% improvement in energy efficiency, and more than 30% reduction in communication outage probability.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated learning (FL) requires edge devices to perform local training and exchange information with a parameter server, leading to substantial energy consumption. A critical challenge in practical FL systems is the rapid energy depletion of battery-limited edge devices, which curtails their operational lifespan and affects the learning performance. To address this issue, we apply energy harvesting technique in FL systems to extract ambient energy for continuously powering edge devices. We first establish the convergence bound for the wireless FL system with energy harvesting devices, illustrating that the convergence is impacted by partial device participation and packet drops, both of which depend on the energy supply. To accelerate the convergence, we formulate a joint device scheduling and power control problem and model it as a Markov decision process (MDP). By solving this MDP, we derive the optimal transmission policy and demonstrate that it possesses a monotone structure with respect to the battery and channel states. To overcome the curse of dimensionality caused by the exponential complexity of computing the optimal policy, we propose a low-complexity algorithm, which is asymptotically optimal as the number of devices increases. Furthermore, for unknown channels and harvested energy statistics, we develop a structure-enhanced deep reinforcement learning algorithm that leverages the monotone structure of the optimal policy to improve the training performance. Finally, extensive numerical experiments on real-world datasets are presented to validate the theoretical results and corroborate the effectiveness of the proposed algorithms.

Problem

• Address energy depletion in battery-limited edge devices for FL.
• Optimize device scheduling and power control using MDP framework.
• Propose low-complexity algorithms for scalable and efficient FL systems.

Innovation

• Energy harvesting for continuous power supply
• MDP framework for optimal transmission policy
• Low-complexity asymptotically optimal algorithm