Discrete Causal Representation Learning

2026-03-26

AI Summary

This work addresses the challenge of identifying causal relationships among a large number of discrete latent variables from low-level, noisy, and mixed-type observations—a task where existing methods often rely on strong assumptions or lack interpretability. The authors propose a discrete causal representation learning framework that models the causal structure among latent variables using a directed acyclic graph (DAG) and links these latents to heterogeneous observations (continuous, count, and binary) via a sparse bipartite graph. Their approach employs a three-stage estimate–resample–discover pipeline, which, without requiring strong structural priors, achieves identifiability of both the latent causal graph and the measurement model for the first time, with theoretical consistency guarantees. Experiments on educational assessment and synthetic image data demonstrate the method’s ability to recover sparse, interpretable causal structures, confirming its effectiveness and robustness.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Causal representation learning seeks to uncover causal relationships among high-level latent variables from low-level, entangled, and noisy observations. Existing approaches often either rely on deep neural networks, which lack interpretability and formal guarantees, or impose restrictive assumptions like linearity, continuous-only observations, and strong structural priors. These limitations particularly challenge applications with a large number of discrete latent variables and mixed-type observations. To address these challenges, we propose discrete causal representation learning (DCRL), a generative framework that models a directed acyclic graph among discrete latent variables, along with a sparse bipartite graph linking latent and observed layers. This design accommodates continuous, count, and binary responses through flexible measurement models while maintaining interpretability. Under mild conditions, we prove that both the bipartite measurement graph and the latent causal graph are identifiable from the observed data distribution alone. We further propose a three-stage estimate-resample-discovery pipeline: penalized estimation of the generative model parameters, resampling of latent configurations from the fitted model, and score-based causal discovery on the resampled latents. We establish the consistency of this procedure, ensuring reliable recovery of the latent causal structure. Empirical studies on educational assessment and synthetic image data demonstrate that DCRL recovers sparse and interpretable latent causal structures.

Problem

• causal representation learning
• discrete latent variables
• mixed-type observations
• identifiability
• interpretability

Innovation

• Discrete Causal Representation Learning
• Identifiability
• Generative Modeling
• Causal Discovery
• Mixed-type Observations