LLM-Based Generative Retrieval for Snapchat Content Recommendation

2026-07-30

AI Summary

This work addresses key challenges in applying large language models to generative retrieval for short video recommendation, including missing item vocabularies, ineffective identifiers, and stringent latency and cost constraints. The authors propose SnapLGR, a novel system that introduces semantic identifiers (SIDs) integrating multimodal embeddings and collaborative signals, enhanced by contrastive learning based on Personalized PageRank to improve codebook utilization. Vocabulary alignment is achieved through continual pretraining (CPT) and supervised fine-tuning (SFT). Furthermore, the system employs a decentralized Worker-Loop architecture accelerated by TensorRT-LLM for efficient inference. Online A/B experiments demonstrate that SnapLGR significantly outperforms the baseline, yielding a 0.37% increase in View Time and a 0.18% improvement in Deep Sessions, thereby validating its effectiveness and scalability.

Abstract

Pretrained large language models (LLMs) are promising retrieval engines because they combine rich semantic priors, strong sequence modeling capabilities, and favorable scaling behavior. However, turning a pretrained LLM into a generative retriever in production deployment raises several challenges: the model must learn an internal item vocabulary that was absent from pretraining, and generate valid item identifiers under strict latency and cost constraints. We address these challenges through the design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat. The system is built around three main designs. First, we construct semantic identifiers (SIDs) from multimodal item embeddings and enhance them with Personalized PageRank (PPR)-based co-engagement contrastive learning, resulting in improved codebook utilization, reduced collisions, and infused collaborative signal. Second, we use continued pretraining (CPT) to ground the introduced SID tokens before supervised fine-tuning (SFT) on user interaction sequences. Third, we make SnapLGR serving practical through TensorRT-LLM CUDA-backed beam search and a decentralized worker-loop architecture. In a live A/B test, the launched system increased View Time by 0.37%, Time Spent by 0.09%, Deep Sessions by 0.18%, and Deep Sessions Unique User by 0.11% relative to the existing TIGER-style generative retrieval baseline. We then decompose this offline gap under a fixed tokenizer and quantify the gains due to model architecture, scaling, and pretraining. Overall, our deployment shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.

Problem

• generative retrieval
• large language models
• content recommendation
• production deployment
• item vocabulary

Innovation

• Generative Retrieval
• Semantic Identifiers
• Continued Pretraining
• Efficient Serving
• Multimodal Embeddings