Towards Distribution Matching between Collaborative and Language Spaces for Generative Recommendation

2025-04-10

AI Summary

Generative recommender systems suffer from semantic misalignment between language models (LMs) and collaborative filtering (CF) spaces, hindering effective feature alignment. To address this, we propose DMRec—a model-agnostic framework that bridges user interactions and LM outputs via a probabilistic meta-network, and introduces a novel three-stage cross-space distribution alignment mechanism optimized using KL and Jensen–Shannon divergences. This is the first approach to achieve joint distribution matching between CF and language spaces. DMRec supports both LM adaptation and generative modeling of implicit feedback, ensuring plug-and-play compatibility and semantic equivalence across representations. Evaluated on three public benchmarks, DMRec consistently enhances the performance of three distinct generative recommendation models, outperforming state-of-the-art LM-augmented methods. Our results empirically validate that explicit distribution alignment significantly improves generative recommendation—demonstrating both effectiveness and broad applicability.

Technology Category

Data Mining & Knowledge Management: Recommender SystemsNatural Language Processing: GenerationMachine Learning: Deep Generative Models & Autoencoders

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Generative recommendation aims to learn the underlying generative process over the entire item set to produce recommendations for users. Although it leverages non-linear probabilistic models to surpass the limited modeling capacity of linear factor models, it is often constrained by a trade-off between representation ability and tractability. With the rise of a new generation of generative methods based on pre-trained language models (LMs), incorporating LMs into general recommendation with implicit feedback has gained considerable attention. However, adapting them to generative recommendation remains challenging. The core reason lies in the mismatch between the input-output formats and semantics of generative models and LMs, making it challenging to achieve optimal alignment in the feature space. This work addresses this issue by proposing a model-agnostic generative recommendation framework called DMRec, which introduces a probabilistic meta-network to bridge the outputs of LMs with user interactions, thereby enabling an equivalent probabilistic modeling process. Subsequently, we design three cross-space distribution matching processes aimed at maximizing shared information while preserving the unique semantics of each space and filtering out irrelevant information. We apply DMRec to three different types of generative recommendation methods and conduct extensive experiments on three public datasets. The experimental results demonstrate that DMRec can effectively enhance the recommendation performance of these generative models, and it shows significant advantages over mainstream LM-enhanced recommendation methods.

Problem

• Bridging generative models and language models for recommendation
• Aligning feature spaces between collaborative and language models
• Enhancing generative recommendation with distribution matching

Innovation

• Model-agnostic framework bridges LM outputs
• Probabilistic meta-network aligns feature spaces
• Cross-space distribution matching maximizes information