🤖 AI Summary
This work addresses the limited semantic interpretability of embeddings in conventional latent factor recommender models, such as matrix factorization, which undermines system transparency and controllability. The authors propose the first application of Matryoshka Sparse Autoencoders (MSAE) to collaborative filtering for learning hierarchical, sparse, and disentangled latent representations. Evaluated on the Amazon Fashion dataset, the approach demonstrates strong semantic coherence of extracted features through alignment with item metadata and automatic annotation via large language models. Furthermore, neuron-level interventions—such as manipulating gender-related latent variables—validate the model’s interpretability and controllability. The method effectively mitigates feature splitting and composition issues commonly observed in traditional sparse autoencoders when scaled to larger embedding dimensions.
📝 Abstract
Latent factor models such as matrix factorization are widely used in recommender systems, yet the learned embedding dimensions typically lack explicit semantic interpretation. This opacity limits transparency, explainability, and principled intervention in recommendation behavior. While sparse autoencoders (SAEs) have recently been used to extract monosemantic features from dense neural representations, standard SAEs suffer from scaling pathologies including feature splitting, feature absorption, and feature composition, which degrade interpretability as dictionary size increases. In this work, we investigate whether hierarchical sparse representations can reveal interpretable structure in collaborative filtering embeddings. We train a large-scale matrix factorization recommender system on the Amazon Fashion dataset and apply a Matryoshka Sparse Autoencoder (MSAE) to the learned embeddings. We analyze the resulting latent features through metadata alignment and LLM-generated labeling to assess semantic coherence and disentanglement. Finally, we show an intervention on a subset of gender associated latent neurons that emerged from the analysis. Our findings suggest that collaborative filtering embeddings contain recoverable hierarchical structure, and that Matryoshka training provides a principled mechanism for exposing interpretable latent factors in interaction-driven recommendation models.