Causal Discovery with Inverted Self-attention for Multivariate Time Series

2026-07-30

AI Summary

This work addresses the challenges of causal discovery in multivariate time series, where high dimensionality, nonlinearity, and complex interactions hinder accurate identification of causal structures. To overcome these limitations, the authors propose a Transformer-based Causal Self-Attention Mechanism (CSAM) that inverts token ordering and induces attention sparsity to prioritize critical causal interactions. The framework further integrates a global causal metric with a causal validation module to effectively capture both latent and indirect causal relationships. Extensive experiments demonstrate that the proposed method consistently outperforms existing approaches across diverse linear and nonlinear datasets. Ablation studies and sensitivity analyses confirm the model’s effectiveness and robustness, highlighting its capacity to recover accurate causal graphs under varying conditions.

Abstract

Causal discovery in multivariate time series data is challenging due to complex interactions, high dimensionality, and nonlinear dependencies among variables. Existing methods often struggle to capture these complexities, resulting in inaccurate causal structures. To address this issue, we propose a novel framework that leverages self-attention mechanisms within the transformer architecture for causal discovery. Our approach introduces a novel inverted causal self-attention mechanism (CSAM) that emphasizes latent and indirect causal relationships by inverting tokens and inducing sparsity in attention scores, focusing on significant causal interactions and reducing spurious correlations. Additionally, we develop a global causal algorithm to identify global causal links, providing a holistic metric for causal influence, along with a causal verification module to ensure robustness in the identified causal relationships, enhancing the reliability of our framework. Experiments on both linear and nonlinear datasets, along with ablation studies and sensitivity analyses, show that our framework outperforms existing methods, demonstrating its potential for causal discovery in complex multivariate time series.

Problem

• causal discovery
• multivariate time series
• nonlinear dependencies
• high dimensionality
• complex interactions

Innovation

• inverted self-attention
• causal discovery
• multivariate time series
• sparsity
• transformer architecture