Technology Category
Machine Learning: Learning with ManifoldsComputer Vision: Interpretability, Explainability, and TransparencyNatural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
LAVA: Explainability for Unsupervised Latent Embeddings
🤖 AI Summary
Unsupervised black-box manifold learning models—particularly those lacking explicit mapping functions—produce latent embeddings that lack interpretability: while preserving relative geometric structure, they fail to establish semantic associations between input features and local embedding organization. To address this, we propose Locality-Aware Variable Associations (LAVA), the first model-agnostic, localized, and reproducible feature association explanation framework specifically designed for mapping-free manifold learning. LAVA integrates UMAP embeddings, local neighborhood modeling, and multivariate correlation analysis to automatically identify salient combinations of input features characterizing distinct regions in the embedding space. Evaluated on MNIST and single-cell kidney datasets, LAVA consistently uncovers locally coherent patterns with clear visual and biological meaning, demonstrating robust reproducibility across distant embedding regions. By enabling principled, region-specific interpretation, LAVA significantly enhances the interpretability of high-dimensional unsupervised representations.
Technology Category
Application Category
📝 Abstract
Unsupervised black-box models can be drivers of scientific discovery, but remain difficult to interpret. Crucially, discovery hinges on understanding the model output, which is often a multi-dimensional latent embedding rather than a well-defined target. While explainability for supervised learning usually seeks to uncover how input features are used to predict a target, its unsupervised counterpart should relate input features to the structure of the learned latent space. Adaptations of supervised model explainability for unsupervised learning provide either single-sample or dataset-wide summary explanations. However, without automated strategies of relating similar samples to one another guided by their latent proximity, explanations remain either too fine-grained or too reductive to be meaningful. This is especially relevant for manifold learning methods that produce no mapping function, leaving us only with the relative spatial organization of their embeddings. We introduce Locality-Aware Variable Associations (LAVA), a post-hoc model-agnostic method designed to explain local embedding organization through its relationship with the input features. To achieve this, LAVA represents the latent space as a series of localities (neighborhoods) described in terms of correlations between the original features, and then reveals reoccurring patterns of correlations across the entire latent space. Based on UMAP embeddings of MNIST and a single-cell kidney dataset, we show that LAVA captures relevant feature associations, with visually and biologically relevant local patterns shared among seemingly distant regions of the latent spaces.
Problem
Research questions and friction points this paper is trying to address.
Interpreting unsupervised black-box models with multi-dimensional latent embeddings
Relating input features to the structure of learned latent space organization
Explaining local embedding neighborhoods through recurring feature correlation patterns
Innovation
Methods, ideas, or system contributions that make the work stand out.
Post-hoc model-agnostic method for local embedding organization
Represents latent space as localities with feature correlations
Reveals reoccurring correlation patterns across entire latent space
🔎 Similar Papers
No similar papers found.
