representation disentanglement

This paper addresses unsupervised representation learning for sequential data. We propose a novel probabilistic flow decomposition framework that disentangles the latent-space dynamics into two orthogonal vector fields: a sparse curl-free field (corresponding to an irrotational potential field) and a divergence-free field (corresponding to a solenoidal rotational field), with sparsity priors newly imposed on both components. Within a variational autoencoder framework, our method jointly optimizes representation encoding, velocity field estimation, and field-structure inference, implicitly learning approximately equivariant representations. Compared to prior approaches, our model simultaneously achieves static representation disentanglement and independence of dynamic transformation primitives, yielding significant improvements in data likelihood and unsupervised equivariance error across multiple sequence transformation benchmarks—achieving state-of-the-art performance. Crucially, the learned vector fields admit clear physical interpretations grounded in classical vector calculus.

This work addresses identity drift and cross-modal inconsistency in controllable multimodal generation, which arise from the lack of explicit structure in semantic attribute evolution. To this end, the authors propose Controlla, a novel framework that, for the first time, integrates graph priors with optimal transport to construct a structured latent space. By imposing graph-based constraints, Controlla explicitly aligns identity and attribute factors, guiding attribute evolution along consistent geometric trajectories. The method introduces a geometry-aware metric to evaluate trajectory consistency and disentanglement and establishes AffectHuman-43K, a leakage-resistant multimodal benchmark. Experimental results demonstrate that Controlla significantly outperforms existing approaches in controllability, identity preservation, and cross-modal alignment, confirming its advantages in graph sensitivity, scalability, and robustness.

This paper addresses the challenge of applying Invariant Risk Minimization (IRM) in fully unsupervised settings, where no labeled data is available. To this end, we propose the first unsupervised IRM framework. Our method comprises two key components: (1) an unsupervised structural causal model that formally characterizes invariance under environmental shifts; and (2) a dual-path algorithm—PICA (Principal Invariant Component Analysis) and VIAE (Variational Interventional Autoencoder)—that achieves identifiable disentanglement of environment-invariant (causal) factors from environment-dependent (spurious) factors. PICA leverages PCA and invariant direction estimation, while VIAE employs a variational autoencoder with generative intervention modeling. Experiments on synthetic benchmarks and environment-shifted MNIST demonstrate that the learned representations capture true causal structure without supervision, significantly improving zero-shot cross-environment generalization and preserving semantic interpretability.

This study addresses the inherent non-identifiability of latent variables in factor models—manifested as non-uniqueness and distributional shifts—by systematically elucidating their nature in linear factor models and their implications for representation learning, drawing on an interdisciplinary perspective spanning psychometrics, statistics, and artificial intelligence. It establishes a theoretical connection between this identifiability issue and posterior collapse in variational autoencoders. By integrating factor analysis, linear autoencoders, and variational inference within a high-dimensional asymptotic framework, the work proves that latent factors become fully identifiable as the observation dimension tends to infinity. Building on this result, the authors propose a nearly distribution-free estimation method for high-dimensional settings, effectively bridging the theoretical gap between classical factor analysis and modern deep generative models, particularly well-suited for representation learning with ultra-high-dimensional data.

To address the lack of high-precision joint position sensing in low-cost robots, this paper proposes a lightweight vision-based proprioceptive method leveraging a single external camera image. Unlike conventional approaches relying on expensive proprioceptive sensors, our method employs a fine-tuned regression architecture requiring only one forward pass, systematically comparing and fusing multiple representations—including CNNs, VAEs, Vision Transformers (ViTs), and uncalibrated fiducial markers—to enable efficient joint state estimation under few-shot (data-scarce) conditions. Evaluated on a 6-DOF low-cost robotic platform, the method achieves sub-centimeter pose estimation accuracy, significantly outperforming model-free baselines. The core contributions are: (i) the first systematic evaluation of diverse representation learning paradigms for single-view visual proprioception, and (ii) the design of an end-to-end deployable framework optimized for resource-constrained edge devices.

This study investigates how Transformers organize their internal representations during next-token prediction pretraining to reflect the underlying structure of the world. By constructing synthetic sequential data with known latent factors and employing geometric activation analysis, subspace dimensionality estimation, and modeling of contextual embedding distributions, the authors find that Transformers exhibit an inductive bias toward decomposing inputs into orthogonal low-dimensional subspaces. When conditional independence holds among latent factors, the model learns lossless factorized representations. Remarkably, even in the presence of noise or hidden dependencies, such structured representations are prioritized during early training stages. These findings reveal fundamental principles governing representational formation in Transformers and demonstrate their inherent preference for factorized structures that mirror compositional aspects of the environment.

This work addresses the interpretability challenge in cross-dataset image generation by proposing unsupervised Contrastive Analysis (CA): a novel paradigm that automatically disentangles shared generative factors from dataset-specific factors between two image collections—without requiring attribute annotations. We formally define and solve the CA problem under weak supervision for the first time. Our general-purpose framework supports both GANs and diffusion models. Key technical innovations include multi-scale feature alignment, discriminative latent-space regularization, and a novel disentanglement loss, jointly optimizing factor separation quality and generation fidelity. Extensive experiments on face, animal, and medical imaging datasets demonstrate state-of-the-art disentanglement accuracy and high-fidelity synthesis, significantly outperforming existing conditional editing and unsupervised disentanglement approaches.

In intelligent agents, the tight coupling between sensory representation learning and decision learning leads to poor generalization and downstream-task-dependent evaluation. Method: We propose Perception Learning (PeL), the first framework to formally decouple perception from decision learning. PeL defines task-agnostic perceptual desiderata—stability, informativeness, and geometric controllability—and optimizes sensory encoders via unlabeled task signals. Theoretically, we prove that representation updates preserving sufficient invariance are orthogonal to the Bayesian risk gradient, ensuring perceptual optimization does not degrade decision performance. Contribution/Results: PeL introduces representation-invariant objective metrics and a task-agnostic evaluation protocol, enabling certified, quantitative assessment of perceptual quality. Experiments demonstrate significant improvements in sensory representation quality without increasing task risk—establishing the first verifiable, evaluable paradigm for perception learning.

Neural networks exhibit high sensitivity to semantically irrelevant transformations—such as ECG phase shifts or IMU sensor rotations—leading to representation degradation and performance collapse. To address this, we propose a structured contrastive learning framework that pioneers the disentanglement of latent representations into three functionally distinct subspaces: invariant (encoding semantics), variant (modeling controlled transformations), and free (capturing residual variability). This design jointly ensures semantic invariance and explicit modeling of admissible variations, unifying robustness with interpretability. Our method requires no architectural modifications; instead, it achieves end-to-end structural learning via semantic grouping and embedded training, advancing contrastive learning from passive data augmentation toward active structural modeling. Evaluated on ECG phase-invariance tasks, our approach achieves a similarity score of 0.91 (+0.66 improvement); on IMU pose-robust activity recognition, it attains 86.65% accuracy and 95.38% rotation consistency.

Current approaches struggle to extract interpretable low-dimensional representations from sparse or incomplete similarity data, limiting our understanding of representational structures in neural, behavioral, and artificial intelligence systems. This work proposes Similarity Representation Factorization (SRF), a novel method that integrates non-negative matrix factorization with low-dimensional embedding to enable, for the first time, generalizable and interpretable extraction of representational dimensions. SRF effectively recovers task-specific model dimensions, accurately predicts independent behavioral attributes, and substantially enhances both exploratory analysis capabilities and statistical power in hypothesis testing. The method is broadly applicable to heterogeneous, multi-source similarity data, offering a robust framework for uncovering latent structure across diverse domains.