🤖 AI Summary
This study addresses the challenges of latent variable identifiability and poor interpretability in high-dimensional single-cell multi-omics data. Methodologically, we systematically investigate the capacity of sparse autoencoders (SAEs) to disentangle latent variables and uncover underlying biological mechanisms. We first validate SAEs’ ability to recover ground-truth generative latents using controllable synthetic data; then perform end-to-end modeling and interpretability analysis on real single-cell multi-omics datasets. Our key contributions are: (1) the first empirical demonstration that SAEs, in a fully unsupervised setting, automatically identify biologically meaningful processes—such as CO₂ transport and ion homeostasis; (2) strong biological consistency in discovering erythroid differentiation and immune regulatory pathways; and (3) a theoretical advance overcoming the identifiability limitation of overparameterized models. Collectively, this work establishes a novel paradigm for interpretable representation learning in multi-omics.
📝 Abstract
Sparse autoencoders (SAEs) have lately been used to uncover interpretable latent features in large language models. Here, we explore their potential for decomposing latent representations in complex and high-dimensional biological data, where the underlying variables are often unknown. On simulated data we show that generative hidden variables can be captured in learned representations in the form of superpositions. The degree to which they are learned depends on the completeness of the representations. Superpositions, however, are not identifiable if these generative variables are unknown. SAEs can to some extent recover these variables, yielding interpretable features. Applied to single-cell multi-omics data, we show that an SAE can uncover key biological processes such as carbon dioxide transport and ion homeostasis, which are crucial for red blood cell differentiation and immune function. Our findings highlight how SAEs can be used in advancing interpretability in biological and other scientific domains.