Integrating the Expected Future in Load Forecasts with Contextually Enhanced Transformer Models

2024-08-26

AI Summary

Existing load forecasting methods struggle to jointly model dynamic forward-looking contextual information—such as user reservations and event schedules—with historical time-series data, resulting in frequent extreme prediction errors and high intraday trading costs. To address this, we propose a context-enhanced sequence-to-sequence Transformer framework that, for the first time, formalizes prospective information as learnable conditional inputs. Our approach introduces dynamic conditional embedding, multi-source temporal alignment, and fusion to jointly encode historical load patterns and planned activity signals. This breaks reliance on static or lagged features inherent in conventional models, significantly improving prediction robustness. Evaluated on national railway energy consumption forecasting, our method reduces MAE by 26.6% over strong baselines. When occupancy reservation data is incorporated in building-level scenarios, MAE further drops by 56.3%, substantially outperforming state-of-the-art approaches.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Time-Series/Data Streams
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Accurate and reliable energy forecasting is essential for power grid operators who strive to minimize extreme forecasting errors that pose significant operational challenges and incur high intra-day trading costs. Incorporating planning information -- such as anticipated user behavior, scheduled events or timetables -- provides substantial contextual information to enhance forecast accuracy and reduce the occurrence of large forecasting errors. Existing approaches, however, lack the flexibility to effectively integrate both dynamic, forward-looking contextual inputs and historical data. In this work, we conceptualize forecasting as a combined forecasting-regression task, formulated as a sequence-to-sequence prediction problem, and introduce contextually-enhanced transformer models designed to leverage all contextual information effectively. We demonstrate the effectiveness of our approach through a primary case study on nationwide railway energy consumption forecasting, where integrating contextual information into transformer models, particularly timetable data, resulted in a significant average mean absolute error reduction of 26.6%. An auxiliary case study on building energy forecasting, leveraging planned office occupancy data, further illustrates the generalizability of our method, showing an average reduction of 56.3% in mean absolute error. Compared to other state-of-the-art methods, our approach consistently outperforms existing models, underscoring the value of context-aware deep learning techniques in energy forecasting applications.

Problem

• Enhancing energy forecasting accuracy
• Integrating contextual and historical data
• Reducing mean absolute error significantly

Innovation

• Contextually-enhanced transformer models
• Sequence-to-sequence prediction formulation
• Integration of planning information