QARM V2: Quantitative Alignment Multi-Modal Recommendation for Reasoning User Sequence Modeling

2026-02-09

AI Summary

Traditional recommender systems relying on ID embeddings suffer from low information density, knowledge isolation, and limited generalization. Conversely, the semantic representations of large language models (LLMs) often misalign with recommendation objectives and cannot be optimized end-to-end. To address these limitations, this work proposes QARM V2, a novel framework that achieves, for the first time, quantitative alignment between LLM-derived semantic representations and recommendation-specific business goals while enabling end-to-end joint training. QARM V2 integrates LLM embeddings, a multimodal alignment mechanism, user sequence modeling, and GSU/ESU architectures, augmented with a learnable quantitative alignment module. This design substantially enhances information density and model generalization, yielding more accurate personalized recommendations in industrial-scale scenarios.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

With the evolution of large language models (LLMs), there is growing interest in leveraging their rich semantic understanding to enhance industrial recommendation systems (RecSys). Traditional RecSys relies on ID-based embeddings for user sequence modeling in the General Search Unit (GSU) and Exact Search Unit (ESU) paradigm, which suffers from low information density, knowledge isolation, and weak generalization ability. While LLMs offer complementary strengths with dense semantic representations and strong generalization, directly applying LLM embeddings to RecSys faces critical challenges: representation unmatch with business objectives and representation unlearning end-to-end with downstream tasks. In this paper, we present QARM V2, a unified framework that bridges LLM semantic understanding with RecSys business requirements for user sequence modeling.

Problem

• recommendation systems
• user sequence modeling
• large language models
• semantic representation
• representation alignment

Innovation

• Quantitative Alignment
• Multi-Modal Recommendation
• LLM-based RecSys
• User Sequence Modeling
• Semantic Representation