A Multi-Armed Bandit Framework for Online Optimisation in Green Integrated Terrestrial and Non-Terrestrial Networks

2025-06-10

AI Summary

To address the high energy consumption and heavy load caused by terrestrial network densification, this paper proposes an online joint optimization framework for green space-air-ground integrated networks, leveraging Multi-Armed Bandit (MAB) theory to dynamically coordinate bandwidth allocation, user association, and macro-base-station sleep scheduling. Its key contribution is the first application of Bandit-feedback Constrained Online Mirror Descent (BCOMD) to energy-efficient optimization in terrestrial–non-terrestrial network (TN–NTN) integration—enabling adaptive trade-offs between energy efficiency and capacity without prior system models and under low-bandwidth feedback constraints. Experimental results demonstrate a significant reduction in the proportion of dissatisfied users during peak hours; during off-peak periods, throughput increases by 19% while energy consumption decreases by 5%, outperforming the 3GPP baseline configuration.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Online Learning & Bandits

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Integrated terrestrial and non-terrestrial network (TN-NTN) architectures offer a promising solution for expanding coverage and improving capacity for the network. While non-terrestrial networks (NTNs) are primarily exploited for these specific reasons, their role in alleviating terrestrial network (TN) load and enabling energy-efficient operation has received comparatively less attention. In light of growing concerns associated with the densification of terrestrial deployments, this work aims to explore the potential of NTNs in supporting a more sustainable network. In this paper, we propose a novel online optimisation framework for integrated TN-NTN architectures, built on a multi-armed bandit (MAB) formulation and leveraging the Bandit-feedback Constrained Online Mirror Descent (BCOMD) algorithm. Our approach adaptively optimises key system parameters--including bandwidth allocation, user equipment (UE) association, and macro base station (MBS) shutdown--to balance network capacity and energy efficiency in real time. Extensive system-level simulations over a 24-hour period show that our framework significantly reduces the proportion of unsatisfied UEs during peak hours and achieves up to 19% throughput gains and 5% energy savings in low-traffic periods, outperforming standard network settings following 3GPP recommendations.

Problem

• Optimize TN-NTN networks for energy efficiency and capacity
• Balance bandwidth allocation and UE association dynamically
• Reduce unsatisfied UEs and improve throughput sustainably

Innovation

• Multi-armed bandit for TN-NTN optimization
• BCOMD algorithm for real-time parameter tuning
• Adaptive bandwidth and MBS shutdown control