Diffusion Language Model for Recommendation

2026-07-23

AI Summary

This work addresses the limitations of existing autoregressive large language model–based recommendation approaches, which overly rely on sequential order, struggle to capture structural item dependencies in user preferences, and suffer from error propagation and underutilized contextual information. To overcome these issues, the paper introduces DLMRec, the first framework to adapt discrete diffusion language models to recommender systems. DLMRec employs a collaboration-aware stochastic tokenizer to model high-order item relationships, integrates a curriculum-driven progressive denoising training strategy, and incorporates a stability-aware voting mechanism to aggregate multi-round predictions. By moving beyond the autoregressive paradigm, this approach significantly enhances recommendation consistency, robustness, and accuracy, effectively capturing complex structures in user preferences.

Abstract

Large language model (LLM)-empowered recommender systems have emerged as a promising paradigm for generative recommendation, leveraging their strong semantic reasoning and generative capacity to model complex, diverse user preferences. However, most existing approaches rely on an autoregressive paradigm that is suboptimal for recommendation. The next-token objective emphasizes sequential order rather than the structural inter-item dependencies underlying user preferences. In addition, prefix-constrained generation restricts bidirectional context and commits to left-to-right decoding, causing early errors to accumulate without correction. Inspired by the success of diffusion language models, we propose \textbf{DLMRec}, a discrete diffusion language model tailored for recommendation that offers a compelling alternative to autoregressive generation. Specifically, DLMRec introduces three key components to bridge diffusion language modeling with recommendation. First, a collaborative-aware stochastic tokenizer encodes multi-hop collaborative signals into expressive discrete tokens compatible with diffusion modeling. Second, a curriculum-driven training strategy aligns the denoising process with preference recovery through progressive item- and token-level learning. Third, a stability-aware voting mechanism aggregates iterative predictions to improve generation consistency and robustness.

Problem

• autoregressive generation
• structural inter-item dependencies
• prefix-constrained generation
• error accumulation
• generative recommendation

Innovation

• diffusion language model
• generative recommendation
• discrete tokenization
• curriculum-driven training
• stability-aware voting