PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction

2026-07-30

AI Summary

Existing semantic identifier–based approaches for multimodal CTR prediction struggle to simultaneously preserve embedding semantic coherence, fine-grained continuous signals, and scalable hierarchical identifiers. To address this, this work proposes PaletteID, inspired by color palettes, which constructs a set of carefully selected prototype items as semantic anchors to bridge pretrained multimodal representations with recommendation models. These prototypes are chosen via a Semantic Quality-aware Determinantal Point Process (SQ-DPP) to balance local density and global diversity. PaletteID then employs a retrieval-aggregation mechanism to generate identifier representations that are interpretable, robust, and scalable. Experiments on two public datasets demonstrate that PaletteID significantly improves CTR prediction performance—particularly for long-tail items—while achieving more stable identifier assignments and enhanced semantic interpretability.

Abstract

Multimodal information can improve the accuracy of click-through rate (CTR) prediction and effectively alleviate item cold-start and long-tail problems. Recent studies commonly discretize pretrained multimodal embeddings into semantic identifiers (SIDs), allowing the model to learn task-specific semantic representations for recommendation. However, existing methods still provide limited gains due to two major limitations. First, codebook assignment fails to preserve semantic relevance and discards fine-grained continuous signals in the original embedding space. Second, the residual code paths are highly dependent on prefix codes, which limits the effective representational scalability of hierarchical identifiers. To address these issues, we propose PaletteID (PID), a prototype-based semantic identifier. Inspired by palette-based color composition, PID uses a compact set of representative prototype items as semantic anchors to bridge pretrained multimodal content space and recommendation models. Specifically, we first construct a prototype palette with Semantic Quality-Aware Determinantal Point Process (SQ-DPP), which jointly considers local content density and global semantic diversity. Then, for each target item, PID retrieves a sequence of semantically related prototypes and aggregates them into an informative PID representation, enabling rich and complementary semantic modeling. Extensive experiments on two public datasets demonstrate that PID consistently improves CTR prediction and yields larger gains for long-tail items. PID also produces more robust identifier assignments and provides more interpretable token semantics than existing residual SID methods.

Problem

• semantic identifiers
• multimodal CTR prediction
• codebook assignment
• residual code paths
• embedding discretization

Innovation

• Prototype-based Semantic Identifier
• Multimodal CTR Prediction
• Determinantal Point Process
• Long-tail Recommendation
• Semantic Composition