CaLR: Causal Latent Revision for Robust Diffusion Reasoning

2026-09-17

AI Summary

为解决自回归模型局部贪婪及扩散语言模型缺乏严格因果结构的问题，提出Causal Latent Revision框架，通过因果拓扑矩阵和隐式微分实现逻辑一致性的梯度引导修正。

Abstract

Autoregressive (AR) models suffer from local greediness, while diffusion language models (DLMs) often lack the strict causal structure required for reasoning. To combine the advantages and overcome the drawbacks of the dual, we propose Causal Latent Revision (CaLR), a framework that reformulates reasoning as constrained latent optimization. By adopting a causal topology matrix (CTM) from an expert model and implicit differentiation, CaLR performs gradient-guided ``thought revision" to enforce logical consistency, enabling dynamic self-correction of intermediate steps during parallel generation. Empirically, CaLR achieves SOTA DLM performance on complex benchmarks, surpassing strong AR baselines and demonstrating superior robustness in constrained tasks like Sudoku.

Problem

• autoregressive models
• diffusion language models
• causal structure
• reasoning

Innovation

• Causal Latent Revision
• causal topology matrix
• implicit differentiation
• thought revision
• robustness