ReeM: Ensemble Building Thermodynamics Model for Efficient HVAC Control via Hierarchical Reinforcement Learning

2025-05-05

AI Summary

Existing building thermodynamic models rely heavily on long-term historical data and domain expertise, resulting in poor generalizability and limited transferability—hindering their applicability to real-time HVAC control. To address this, we propose a thermodynamic model integration framework specifically designed for HVAC control. Our approach introduces a novel hierarchical reinforcement learning (HRL)-based mechanism for dynamic model selection and online weighted ensemble, enabling adaptive modeling of non-stationary building time-series data. Leveraging pre-existing models as foundational components, the framework eliminates the need for de novo modeling, thereby significantly enhancing cross-building transferability and modeling efficiency. Offline evaluations and on-site deployment demonstrate that our method reduces prediction error by 32%, decreases modeling time by 76%, and achieves a 14.8% energy saving in HVAC operation.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The building thermodynamics model, which predicts real-time indoor temperature changes under potential HVAC (Heating, Ventilation, and Air Conditioning) control operations, is crucial for optimizing HVAC control in buildings. While pioneering studies have attempted to develop such models for various building environments, these models often require extensive data collection periods and rely heavily on expert knowledge, making the modeling process inefficient and limiting the reusability of the models. This paper explores a model ensemble perspective that utilizes existing developed models as base models to serve a target building environment, thereby providing accurate predictions while reducing the associated efforts. Given that building data streams are non-stationary and the number of base models may increase, we propose a Hierarchical Reinforcement Learning (HRL) approach to dynamically select and weight the base models. Our approach employs a two-tiered decision-making process: the high-level focuses on model selection, while the low-level determines the weights of the selected models. We thoroughly evaluate the proposed approach through offline experiments and an on-site case study, and the experimental results demonstrate the effectiveness of our method.

Problem

• Predicts indoor temperature changes for HVAC control optimization
• Reduces data and expert reliance via model ensemble approach
• Dynamically selects and weights models using HRL for accuracy

Innovation

• Ensemble model for HVAC control optimization
• Hierarchical Reinforcement Learning for dynamic selection
• Two-tiered decision-making for model weighting