HyMiRec: A Hybrid Multi-interest Learning Framework for LLM-based Sequential Recommendation

2025-10-15

AI Summary

Existing LLM-based sequential recommendation methods face two key bottlenecks: (1) truncating long user interaction sequences due to prohibitive inference overhead, thereby losing long-term preferences; and (2) relying on a single prediction embedding, which limits modeling of users’ multifaceted interests. To address these, we propose HyMiRec—a hybrid multi-interest learning framework. It first extracts coarse-grained long-term interests via a lightweight recommendation module, then leverages an LLM to model fine-grained, dynamic preferences. A cosine-similarity-driven residual codebook enables efficient compression and reuse of historical embeddings. Furthermore, a decoupled multi-interest query mechanism adaptively separates and represents heterogeneous interest signals. Extensive experiments on multiple public and industrial datasets demonstrate significant improvements over state-of-the-art methods. Online A/B tests confirm consistent gains in both click-through rate and recommendation diversity.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Recommender Systems
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) have recently demonstrated strong potential for sequential recommendation. However, current LLM-based approaches face critical limitations in modeling users'long-term and diverse interests. First, due to inference latency and feature fetching bandwidth constraints, existing methods typically truncate user behavior sequences to include only the most recent interactions, resulting in the loss of valuable long-range preference signals. Second, most current methods rely on next-item prediction with a single predicted embedding, overlooking the multifaceted nature of user interests and limiting recommendation diversity. To address these challenges, we propose HyMiRec, a hybrid multi-interest sequential recommendation framework, which leverages a lightweight recommender to extracts coarse interest embeddings from long user sequences and an LLM-based recommender to captures refined interest embeddings. To alleviate the overhead of fetching features, we introduce a residual codebook based on cosine similarity, enabling efficient compression and reuse of user history embeddings. To model the diverse preferences of users, we design a disentangled multi-interest learning module, which leverages multiple interest queries to learn disentangles multiple interest signals adaptively, allowing the model to capture different facets of user intent. Extensive experiments are conducted on both benchmark datasets and a collected industrial dataset, demonstrating our effectiveness over existing state-of-the-art methods. Furthermore, online A/B testing shows that HyMiRec brings consistent improvements in real-world recommendation systems.

Problem

• Addresses truncation of user sequences losing long-term preferences
• Solves single-embedding limitation in capturing diverse user interests
• Overcomes feature fetching constraints through efficient embedding compression

Innovation

• Hybrid framework combines lightweight and LLM recommenders
• Residual codebook compresses user history via cosine similarity
• Disentangled multi-interest module adaptively captures diverse preferences