CrossLinear: Plug-and-Play Cross-Correlation Embedding for Time Series Forecasting with Exogenous Variables

2025-05-29

AI Summary

To address modeling bias and overfitting caused by conflation of endogenous and exogenous variables in time series forecasting with exogenous inputs, this paper proposes a lightweight, plug-and-play Cross-Variable Cross-Correlation Embedding (CCE) module. CCE introduces the first time-invariant, direct mechanism for modeling cross-variable dependencies, integrating block-wise temporal processing with a global linear prediction head to jointly capture multi-scale temporal dynamics and inter-variable relationships. Its modular design ensures seamless compatibility with mainstream architectures while substantially mitigating overfitting in dependency modeling. Extensive experiments across 12 real-world datasets demonstrate consistent state-of-the-art performance on both short-term and long-term forecasting tasks. Ablation studies validate the efficacy of each component, and cross-domain evaluation confirms strong generalization capability.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Diffusion Models for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Time series forecasting with exogenous variables is a critical emerging paradigm that presents unique challenges in modeling dependencies between variables. Traditional models often struggle to differentiate between endogenous and exogenous variables, leading to inefficiencies and overfitting. In this paper, we introduce CrossLinear, a novel Linear-based forecasting model that addresses these challenges by incorporating a plug-and-play cross-correlation embedding module. This lightweight module captures the dependencies between variables with minimal computational cost and seamlessly integrates into existing neural networks. Specifically, it captures time-invariant and direct variable dependencies while disregarding time-varying or indirect dependencies, thereby mitigating the risk of overfitting in dependency modeling and contributing to consistent performance improvements. Furthermore, CrossLinear employs patch-wise processing and a global linear head to effectively capture both short-term and long-term temporal dependencies, further improving its forecasting precision. Extensive experiments on 12 real-world datasets demonstrate that CrossLinear achieves superior performance in both short-term and long-term forecasting tasks. The ablation study underscores the effectiveness of the cross-correlation embedding module. Additionally, the generalizability of this module makes it a valuable plug-in for various forecasting tasks across different domains. Codes are available at https://github.com/mumiao2000/CrossLinear.

Problem

• Model dependencies between endogenous and exogenous time series variables
• Prevent overfitting in variable dependency modeling
• Capture both short-term and long-term temporal dependencies

Innovation

• Plug-and-play cross-correlation embedding module
• Patch-wise processing with global linear head
• Lightweight module for variable dependencies