User Long-Term Multi-Interest Retrieval Model for Recommendation

2025-07-14

AI Summary

Industrial recommender systems face challenges in modeling ultra-long user behavior sequences (thousands of interactions) within retrieval models, constrained by real-time latency requirements, lack of target-aware mechanisms, and insufficient cross-interaction architectures. To address this, we propose ULIM—a Unified Long-sequence Interest Modeling framework. Its key innovations are: (1) category-aware dual-interest hierarchical learning, jointly optimizing short- and long-term interests; and (2) a pointer-enhanced two-stage cascaded retrieval architecture (PGIN), enabling target-aware, category-to-item cross-interaction. ULIM is the first retrieval-stage framework to efficiently model ultra-long behavioral sequences. Extensive experiments on the Taobao dataset demonstrate significant improvements over state-of-the-art methods. Deployed online in flash-sale scenarios, ULIM achieves +5.54% CTR, +11.01% order volume, and +4.03% GMV.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

User behavior sequence modeling, which captures user interest from rich historical interactions, is pivotal for industrial recommendation systems. Despite breakthroughs in ranking-stage models capable of leveraging ultra-long behavior sequences with length scaling up to thousands, existing retrieval models remain constrained to sequences of hundreds of behaviors due to two main challenges. One is strict latency budget imposed by real-time service over large-scale candidate pool. The other is the absence of target-aware mechanisms and cross-interaction architectures, which prevent utilizing ranking-like techniques to simplify long sequence modeling. To address these limitations, we propose a new framework named User Long-term Multi-Interest Retrieval Model(ULIM), which enables thousand-scale behavior modeling in retrieval stages. ULIM includes two novel components: 1)Category-Aware Hierarchical Dual-Interest Learning partitions long behavior sequences into multiple category-aware subsequences representing multi-interest and jointly optimizes long-term and short-term interests within specific interest cluster. 2)Pointer-Enhanced Cascaded Category-to-Item Retrieval introduces Pointer-Generator Interest Network(PGIN) for next-category prediction, followed by next-item retrieval upon the top-K predicted categories. Comprehensive experiments on Taobao dataset show that ULIM achieves substantial improvement over state-of-the-art methods, and brings 5.54% clicks, 11.01% orders and 4.03% GMV lift for Taobaomiaosha, a notable mini-app of Taobao.

Problem

• Modeling ultra-long user behavior sequences in recommendation retrieval
• Overcoming latency constraints for real-time large-scale candidate retrieval
• Enhancing target-aware multi-interest learning from diverse user behaviors

Innovation

• Category-Aware Hierarchical Dual-Interest Learning partitions sequences
• Pointer-Enhanced Cascaded Category-to-Item Retrieval predicts categories
• ULIM enables thousand-scale behavior modeling in retrieval