🤖 AI Summary
Existing Transformer-based time series forecasting models overemphasize temporal modeling, resulting in high computational overhead with marginal performance gains and severely constrained representation capacity due to suboptimal embedding design. To address this, we propose Hybrid Temporal–Multivariate Embedding (HTME), a lightweight yet effective embedding framework that jointly captures temporal dynamics and inter-variable dependencies via two synergistic components: a temporal feature extraction module and a multivariate relational modeling module. HTME is plug-and-play compatible with standard Transformer architectures and enhances sequence representation without increasing decoding complexity. Extensive experiments across eight real-world datasets demonstrate that HTME consistently outperforms state-of-the-art baselines—achieving an average 6.2% reduction in MAE and a 31% decrease in FLOPs—thereby validating the fundamental benefit of multivariate-aware embedding for time series modeling.
📝 Abstract
Transformer-based methods have achieved impressive results in time series forecasting. However, existing Transformers still exhibit limitations in sequence modeling as they tend to overemphasize temporal dependencies. This incurs additional computational overhead without yielding corresponding performance gains. We find that the performance of Transformers is highly dependent on the embedding method used to learn effective representations. To address this issue, we extract multivariate features to augment the effective information captured in the embedding layer, yielding multidimensional embeddings that convey richer and more meaningful sequence representations. These representations enable Transformer-based forecasters to better understand the series. Specifically, we introduce Hybrid Temporal and Multivariate Embeddings (HTME). The HTME extractor integrates a lightweight temporal feature extraction module with a carefully designed multivariate feature extraction module to provide complementary features, thereby achieving a balance between model complexity and performance. By combining HTME with the Transformer architecture, we present HTMformer, leveraging the enhanced feature extraction capability of the HTME extractor to build a lightweight forecaster. Experiments conducted on eight real-world datasets demonstrate that our approach outperforms existing baselines in both accuracy and efficiency.