Reinforcement Learning (RL) Meets Urban Climate Modeling: Investigating the Efficacy and Impacts of RL-Based HVAC Control

2025-05-11

AI Summary

Reinforcement learning (RL)-driven HVAC control faces challenges in cross-regional transferability across diverse urban climates, particularly concerning energy efficiency, indoor thermal comfort, and localized urban climate impacts. Method: This study establishes a multi-scale coupled simulation framework integrating RL, building energy modeling (EnergyPlus), urban canopy modeling (UCM), and meteorological data. It pioneers embedding HVAC RL policies within a climate feedback loop to quantify how ambient climate conditions govern reward function sensitivity and policy transferability, introducing the “inter-city learning” paradigm. Contribution/Results: Experiments reveal that hot cities achieve higher rewards under most trade-off configurations; cities with greater temperature variability exhibit stronger policy transferability; and RL deployment necessitates climate-specific evaluation. The work provides theoretical foundations and practical guidelines for climate-adaptive intelligent building control, advancing the design of resilient, energy-efficient urban infrastructure.

Technology Category

• Machine Learning: Reinforcement Learning
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Reinforcement learning (RL)-based heating, ventilation, and air conditioning (HVAC) control has emerged as a promising technology for reducing building energy consumption while maintaining indoor thermal comfort. However, the efficacy of such strategies is influenced by the background climate and their implementation may potentially alter both the indoor climate and local urban climate. This study proposes an integrated framework combining RL with an urban climate model that incorporates a building energy model, aiming to evaluate the efficacy of RL-based HVAC control across different background climates, impacts of RL strategies on indoor climate and local urban climate, and the transferability of RL strategies across cities. Our findings reveal that the reward (defined as a weighted combination of energy consumption and thermal comfort) and the impacts of RL strategies on indoor climate and local urban climate exhibit marked variability across cities with different background climates. The sensitivity of reward weights and the transferability of RL strategies are also strongly influenced by the background climate. Cities in hot climates tend to achieve higher rewards across most reward weight configurations that balance energy consumption and thermal comfort, and those cities with more varying atmospheric temperatures demonstrate greater RL strategy transferability. These findings underscore the importance of thoroughly evaluating RL-based HVAC control strategies in diverse climatic contexts. This study also provides a new insight that city-to-city learning will potentially aid the deployment of RL-based HVAC control.

Problem

• Evaluating RL-based HVAC control efficacy in diverse climates
• Assessing impacts of RL strategies on indoor and urban climates
• Exploring transferability of RL strategies across different cities

Innovation

• RL-based HVAC control for energy efficiency
• Integrated framework with urban climate model
• City-to-city learning enhances strategy transferability