Structure-agnostic Causal Representation Learning

2026-09-30

AI Summary

This study addresses the constraint conflicts and information loss arising from predefined causal structures in causal representation learning. To this end, we propose SaCRL, a framework that jointly identifies causal structures and learns invariant representations without requiring prior knowledge. Methodologically, SaCRL introduces a novel HSIC-based soft optimization mechanism for structure selection, combined with random feature approximation and adaptive weighting techniques to focus on achievable structures, all supported by rigorous theoretical guarantees. Experimental results demonstrate that the framework accurately recovers ground-truth causal structures on synthetic data, significantly outperforms existing baselines on Colored MNIST, and achieves state-of-the-art performance on the DomainBed benchmark, exhibiting superior generalization robustness.

Abstract

Causal representation learning aims to discover robust features by exploiting the causal structure underlying data generation. Existing methods require specifying the causal structure a priori, yet different structures demand fundamentally incompatible invariance constraints, and misspecification leads to representations that discard predictive information. We introduce SaCRL, a framework that jointly identifies the causal structure and learns the corresponding invariant representation without prior structural knowledge. Our approach formulates structure selection as a soft optimization over candidate invariances using HSIC-based violation metrics, with adaptive weights that automatically concentrate on the achievable structure. We provide theoretical guarantees for structure identification, including under random-feature approximation, invariance satisfaction, and out-of-distribution generalization. Empirically, SaCRL recovers the true structure on synthetic and semi-synthetic Bayesian-network benchmarks, outperforms fixed-invariance baselines on Colored MNIST, achieves state-of-the-art accuracy on three DomainBed benchmarks (PACS, VLCS, OfficeHome), and degrades gracefully under structural misspecification and limited environment diversity. Code is available at: https://github.com/ArmanBehnam/sacrl.

Problem

• Causal Representation Learning
• Structure-agnostic
• Invariant Representation
• Structure Misspecification
• Out-of-distribution Generalization

Innovation

• Causal Representation Learning
• Structure-agnostic
• HSIC
• Invariant Representation
• Out-of-distribution Generalization