Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings

2026-06-01

AI Summary

This study addresses the growing complexity of building energy management due to increased integration of renewable energy sources and the limited deployability of conventional deep reinforcement learning (DRL) approaches owing to their lack of interpretability. To bridge this gap, the authors propose an interpretable DRL framework tailored for building energy management that integrates real-time data, external signals, and predictive information. The work systematically evaluates on-policy algorithms—including A2C and PPO—on both real-world and synthetic datasets for battery scheduling performance. Notably, it pioneers the application of post-hoc explainability techniques to DRL policies in building energy contexts. Empirical validation at the Living Lab Energy Campus demonstrates that on-policy methods significantly outperform off-policy counterparts in cumulative reward and policy stability, effectively reducing electricity costs while delivering transparent, actionable decision rationales—thus achieving a unified balance between high performance and high interpretability.

Abstract

The increasing integration of renewable energy sources into power systems, particularly in buildings equipped with photovoltaic (PV) panels and energy storage systems, introduces significant complexity in energy systems. Volatile power generation, varying electricity tariffs, and increased entities, e.g., PV systems, and heat pumps, have increased the complexity and made the system harder to operate. This leads to the demand for additional control and optimization routes including data-based controls, such as reinforcement learning. While deep reinforcement learning (DRL) has emerged as a promising solution to optimize building operations in dynamic and ever more complex environments, its black-box nature impedes user trust and practical adoption. This paper presents a framework for explainable deep reinforcement learning (XRL) applied to energy management in residential buildings. We demonstrate its usage on both synthetic data but also on real-world data from the Living Lab Energy Campus (LLEC) at KIT. We train and compare both on-policy and off-policy DRL agents on an expanded state space that incorporates real-time measurements (demand, PV generation, battery power, state of charge), external signals (dynamic electricity price, local weather data), calendrical and holiday indicators, and forecasts for demand and price. Our experimental results indicate that on-policy algorithms, particularly Advantage Actor Critic (A2C) and Proximal Policy Optimization (PPO), outperform off-policy methods in terms of cumulative rewards and policy stability. To explain these models, we employ post-hoc interpretation techniques to elaborate the learned control policies. Our findings demonstrate that the XRL framework not only reduces electricity costs through optimal battery management, but also provides transparent, actionable insights into the agent's decision-making process.

Problem

• Explainable AI
• Deep Reinforcement Learning
• Energy Management
• Building Automation
• Renewable Energy Integration

Innovation

• Explainable Reinforcement Learning
• Deep Reinforcement Learning
• Energy Management
• On-policy Algorithms
• Post-hoc Interpretation