Learning Sparse Representations of Multimodal Content for Enhanced Cold Item Recommendation

2026-07-19

AI Summary

This work addresses the challenges of cold-start items in large-scale recommender systems, which suffer from a lack of interaction history and the high storage overhead of dense embeddings. To this end, the authors propose a sparse embedding learning framework tailored for cold-start scenarios. By integrating multimodal content information and introducing a pre-sparsification activation mechanism based on linear attention, the method effectively sharpens item similarity and enhances denoising capability, while also supporting multi-interest user modeling. Extensive experiments on four multimodal recommendation benchmarks demonstrate that the proposed approach significantly outperforms existing dense embedding methods, achieving higher cold-start recommendation accuracy with substantially reduced storage costs. Moreover, the framework exhibits strong interpretability and robust trade-offs between accuracy and model scale.

Abstract

The scale and rapid growth of item catalogs in modern digital platforms present significant challenges to recommender system (RS) practitioners. Most RSs use embedding similarity to predict user-item preferences, but embedding storage and low-latency retrieval is challenging in industry-scale catalogs. Furthermore, newly added items do not have corresponding embeddings and cannot be recommended effectively; previous works often tackle this item cold-start problem by generating cold item representations from auxiliary content, such as images or descriptive text, so that user preferences can be predicted without historical interactions. In this paper, we argue that sparse embeddings have notable advantages over standard dense vectors in this content-based cold-start paradigm. We describe how existing cold-start training regimes can be adapted for sparse representation learning, and build on insights from linear attention to design a pre-sparsification activation technique that induces sharpness and denoising effects in learned item-item similarities. We show that the resulting sparse embeddings achieve significant improvements in cold-start recommendation accuracy over dense embeddings at considerably lower storage costs, especially for users with multiple interests. Through comprehensive experiments on four multimodal RS datasets, we also demonstrate the interpretability of sparse content embeddings and their robustness in the trade-off between size and accuracy.

Problem

• cold-start recommendation
• sparse embeddings
• multimodal content
• item representation
• recommender systems

Innovation

• sparse embeddings
• cold-start recommendation
• multimodal content
• pre-sparsification activation
• linear attention