Temporally Encoded Double DQN for Proactive PRB Allocation in O-RAN Enabled Industrial Networks

2026-05-28

AI Summary

This work addresses the challenge posed by temporally correlated and non-stationary traffic in industrial networks, which hinders traditional static or reactive schedulers from ensuring delay reliability and leads to suboptimal resource utilization. To overcome this, the authors propose a temporal-aware deep reinforcement learning xApp that, for the first time, integrates an LSTM encoder into a Double DQN framework to model temporal dependencies of slice-level KPIs. Concurrently, a continuous-time Markov chain (CTMC) is employed to capture traffic concurrency and burstiness, enabling proactive physical resource block allocation within the O-RAN architecture. The proposed method significantly enhances network slice satisfaction and buffer stability under medium-to-high load conditions, with performance gains further increasing as the length of the historical observation window expands.

Abstract

Fifth-generation (5G) wireless systems are increasingly adopted in smart manufacturing to support heterogeneous industrial workloads through services such as enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low-Latency Communication (URLLC). However, industrial traffic is inherently process-driven and temporally correlated. So, static or reactive schedulers in the Open Radio Access Network (O-RAN) are inadequate for such non-stationary conditions, leading to sub-optimal utilization and violation of latency-reliability guarantees. This paper proposes a temporal-aware deep reinforcement learning (DRL) xApp for proactive Physical Resource Block (PRB) allocation in O-RAN-enabled industrial networks. The proposed framework integrates a long short-term memory (LSTM) encoder within a Double Deep Q-Network (DQN) to model sequential dependencies among slice-level Key Performance Indicators (KPIs), enabling predictive and stable decision-making. A continuous-time Markov chain (CTMC) traffic model is incorporated to emulate machine concurrency and process burstiness. Experimental results show that the LSTM-Double DQN improves slice satisfaction, and buffer stability under moderate and heavy load, with the longest sequence window providing the strongest gains.

Problem

• O-RAN
• Industrial Networks
• PRB Allocation
• Temporal Correlation
• Latency-Reliability Guarantees

Innovation

• Temporal-aware DRL
• LSTM-Double DQN
• Proactive PRB Allocation
• O-RAN
• Industrial Traffic Modeling