A Reinforcement Learning Framework for Resource Allocation in Uplink Carrier Aggregation in the Presence of Self Interference

2025-11-22

AI Summary

This paper addresses the joint optimization of carrier activation (discrete) and power allocation (continuous) for power-constrained users in uplink carrier aggregation under dynamic channel and interference conditions, subject to nonlinear self-interference (SI) constraints to preserve downlink receiver sensitivity. We propose an online optimization framework based on a composite-action-space Actor-Critic reinforcement learning architecture. Our method innovatively introduces a hybrid action policy capable of simultaneously generating discrete carrier-activation decisions and continuous power allocations, coupled with a SI-aware reward function tailored to suppress self-interference. This design enables adaptive responses to time-varying channel gains and interference. Experimental results demonstrate that the proposed approach significantly outperforms conventional heuristic and convex-optimization baselines in terms of aggregate throughput, while maintaining high stability and robustness both with and without self-interference.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Online Learning & Bandits
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Carrier aggregation (CA) is a technique that allows mobile networks to combine multiple carriers to increase user data rate. On the uplink, for power constrained users, this translates to the need for an efficient resource allocation scheme, where each user distributes its available power among its assigned uplink carriers. Choosing a good set of carriers and allocating appropriate power on the carriers is important. If the carrier allocation on the uplink is such that a harmonic of a user's uplink carrier falls on the downlink frequency of that user, it leads to a self coupling-induced sensitivity degradation of that user's downlink receiver. In this paper, we model the uplink carrier aggregation problem as an optimal resource allocation problem with the associated constraints of non-linearities induced self interference (SI). This involves optimization over a discrete variable (which carriers need to be turned on) and a continuous variable (what power needs to be allocated on the selected carriers) in dynamic environments, a problem which is hard to solve using traditional methods owing to the mixed nature of the optimization variables and the additional need to consider the SI constraint. We adopt a reinforcement learning (RL) framework involving a compound-action actor-critic (CA2C) algorithm for the uplink carrier aggregation problem. We propose a novel reward function that is critical for enabling the proposed CA2C algorithm to efficiently handle SI. The CA2C algorithm along with the proposed reward function learns to assign and activate suitable carriers in an online fashion. Numerical results demonstrate that the proposed RL based scheme is able to achieve higher sum throughputs compared to naive schemes. The results also demonstrate that the proposed reward function allows the CA2C algorithm to adapt the optimization both in the presence and absence of SI.

Problem

• Optimizing uplink carrier aggregation with power allocation constraints
• Managing self-interference from harmonic frequency overlaps in carriers
• Solving mixed discrete-continuous resource allocation in dynamic environments

Innovation

• Reinforcement learning framework for uplink resource allocation
• Compound-action actor-critic algorithm handling mixed variables
• Novel reward function enabling self-interference constraint adaptation