The Challenges of Using Reinforcement Learning for Controlling Industrial Energy Systems

2026-05-29

AI Summary

This study addresses the persistent performance gap between simulation and real-world deployment of reinforcement learning (RL) in industrial energy systems, using a district heating network as a case study. The control task is formulated as a Markov decision process, and key challenges—including partial observability, action space design, reward function shaping, and sim-to-real transfer—are systematically analyzed. For the first time, this work provides a comprehensive empirical investigation of RL deployment barriers in an actual industrial setting, uncovering the root causes of performance discrepancies. Although the system achieved stable operation in practice, its measured performance fell significantly short of simulation results, thereby validating critical bottlenecks in real-world RL applications and establishing an empirical foundation for future co-optimization of algorithms and engineering design.

Abstract

Reinforcement learning has shown promising results for optimizing the control of industrial energy systems, yet most existing studies remain limited to the application in simulation environments. We investigate the challenges of deploying reinforcement learning in a real-world industrial energy system, considering a thermal heating network as a use case. We formulate the task as a Markov Decision Process and systematically analyze the associated challenges along the structure of the formal description, including partial observability, action space design, reward design, and the simulation-to-reality gap. The challenges are grounded in an existing real-world deployment, where reinforcement learning achieves operational stability but shows a significant performance gap compared to simulation.

Problem

• reinforcement learning
• industrial energy systems
• simulation-to-reality gap
• partial observability
• reward design

Innovation

• reinforcement learning
• industrial energy systems
• simulation-to-reality gap
• Markov Decision Process
• real-world deployment