Causal Semantic Alignment for LLM-based Time Series Forecasting

2026-06-06

AI Summary

This work addresses the spurious correlations in large language models for time series forecasting, which arise from the entanglement of dynamic fluctuations and invariant semantics. To tackle this issue, the authors propose CVAformer, a novel framework that explicitly decouples time series into invariant and dynamic components at the variable level. CVAformer introduces, for the first time, a variable-level semantic alignment and dynamic disentanglement mechanism, combined with causal intervention to eliminate confounding effects from dynamic components. Additionally, it employs a non-causal attention mechanism to enhance modeling of inter-variable interactions. Experimental results demonstrate that CVAformer achieves state-of-the-art or competitive performance across diverse forecasting scenarios—including long-term, short-term, few-shot, and zero-shot settings—and significantly improves prediction accuracy and robustness on multiple benchmark datasets.

Abstract

Recent advances in Large Language Models (LLMs) have opened new possibilities for time series forecasting by enabling alignment between temporal patterns and pretrained word embeddings. However, most LLM-based methods overlook the heterogeneous nature of time series, where dynamic fluctuations and invariant semantics are entangled. This entanglement introduces spurious correlations during the alignment, as dynamic components act as confounders by simultaneously influencing invariant components and the resulting aligned embeddings. To address this issue, a variable-level alignment framework CVAformer is proposed. CVAformer explicitly disentangles each variable into invariant and dynamic components just before alignment, and applies causal intervention to mitigate the confounding effect of the dynamics. To better support variable-level alignment, CVAformer replaces the standard causal attention in LLMs with a non-causal attention mechanism that captures interactions among variables at each time step. Extensive experiments across long-term, short-term, few-shot, and zero-shot forecasting settings indicate that CVAformer matches or exceeds state-of-the-art performance on most datasets, and in some cases achieves notably better accuracy. Experimental results validate the effectiveness of variable-level alignment and dynamic disentanglement in CVAformer, offering a new perspective for LLM-based time series tasks.

Problem

• time series forecasting
• causal alignment
• spurious correlation
• dynamic disentanglement
• heterogeneous time series

Innovation

• causal intervention
• dynamic disentanglement
• variable-level alignment
• non-causal attention
• LLM-based forecasting