Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing

2026-04-14

AI Summary

This work addresses the problem of identifying causal latent variables from high-dimensional observations generated by piecewise affine mixture functions, where the latents follow a degenerate Gaussian mixture distribution. To overcome the challenge that the data density is undefined due to distributional degeneracy, the authors propose a two-stage estimation algorithm: first imposing sparsity and Gaussianity constraints in the learned representation, then explicitly modeling the piecewise affine structure. Theoretically, this study establishes the first asymptotic identifiability guarantees under degenerate Gaussian mixtures and piecewise affine generative assumptions, ensuring consistent recovery of latent variables up to permutation and scaling. Experiments on both synthetic and image datasets demonstrate that the proposed method significantly outperforms existing baselines in accurately recovering the true latent variables.

Technology Category

Machine Learning: Causal LearningReasoning under Uncertainty: Relational Probabilistic ModelsConstraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSemantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semanticsEconomics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method's effectiveness in recovering the ground-truth latent variables.

Problem

• causal representation learning
• Gaussian mixture models
• identifiability
• degenerate distributions
• piecewise affine mixing

Innovation

• identifiability
• degenerate Gaussian mixture models
• piecewise affine mixing
• causal representation learning
• sparsity regularization