Correlation or Causation: Analyzing the Causal Structures of LLM and LRM Reasoning Process

2025-09-22

AI Summary

Existing large language models (LLMs) and language reasoning models (LRMs) often rely on spurious surface-level correlations rather than genuine causal relationships, leading to unfaithful reasoning, hallucinations, and biases. Method: This paper introduces structural causal models (SCMs) into LLM/LRM reasoning analysis for the first time, integrating counterfactual interventions and causal graph modeling to quantitatively characterize causal structures among instructions, chain-of-thought steps, and final answers. We propose RLVR—a novel training paradigm combining reinforcement learning and knowledge distillation—to explicitly strengthen true causal pathways and suppress spurious associations during training. Contribution/Results: Empirical results demonstrate that RLVR-trained LRMs better approximate ideal causal models, significantly improving reasoning faithfulness, consistency, and robustness. All code and data are publicly released.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

LLMs suffer from critical reasoning issues such as unfaithfulness, bias, and inconsistency, since they lack robust causal underpinnings and may rely on superficial correlations rather than genuine understanding. Successive LRMs have emerged as a promising alternative, leveraging advanced training techniques such as reinforcement learning (RL) and distillation to improve task accuracy. However, the impact of these training methods on causality remains largely unexplored. In this study, we conduct a systematic causal analysis on LLMs and LRMs, examining structural causal models (SCMs) of four key variables: problem instruction (Z), thinking process (T), reasoning steps (X), and answer (Y). Our findings reveal that RLVR-trained LRMs exhibit enhanced causal reasoning capabilities, aligning more closely with ideal causal structures, while LLMs and distilled LRMs fail to address causality-related deficiencies. Our further investigation indicates that RLVR reduces spurious correlations and strengthens genuine causal patterns, thereby mitigating unfaithfulness and bias. In addition, our inspection on the dynamics of the RLVR training process observes a high correlation between reduced spurious features and improved causal structures, where the causal relationships consistently improve in the training process. This study contributes to the understanding of causality in reasoning models, highlights the critical role of RLVR in enhancing causal reasoning, and provides insights for designing future AI systems with stronger causal foundations. We release our code and data at https://github.com/Harryking1999/CoT_Causal_Analysis.

Problem

• Analyzing causal structures in LLM and LRM reasoning processes
• Investigating training methods' impact on causality in reasoning models
• Addressing unfaithfulness and bias issues through causal analysis

Innovation

• Used structural causal models for analysis
• Applied RLVR training to enhance causality
• Reduced spurious correlations in reasoning models