RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation

2026-07-28

AI Summary

This work addresses the limitations of existing multimodal large language model–based recommender systems, which struggle to effectively integrate user behavioral signals into item description generation and require user conditioning that complicates deployment. To overcome these challenges, the authors propose RecoReward, a framework that introduces Recommender Affinity Scores (RAS) as reinforcement learning rewards during training. RAS implicitly models universal user preferences by contrasting interactions from historically engaged users against non-target users, thereby guiding semantic generation without explicit user conditioning. At inference time, RecoReward operates solely on content inputs, eliminating the need for user information and enabling efficient deployment. Experimental results demonstrate that RecoReward-9B consistently outperforms strong baselines such as Qwen3.5-9B across seven offline recall metrics and achieves significant gains in online A/B tests.

Abstract

Multimodal large language models (MLLMs) can convert multimodal item content into structured descriptions used as semantic features for recommendation. Conventional content-only generation, however, cannot use downstream user signals to determine which semantics should be emphasized. Recent user-conditioned methods incorporate these signals through user histories or profiles, but they require user information at inference and make generation user-dependent. In this paper, we introduce RecoReward, which instead uses behavior-derived rewards during training and preserves content-only inference. To instantiate this idea in live-stream recommendation, we treat historically engaged users as a proxy for future target users and use observational non-target users to estimate affinity shared broadly across users. The Recommender Affinity Score (RAS) contrasts these signals to provide user-selective feedback for reinforcement learning, allowing the learned policy to generate a single shared description without user inputs. In our offline benchmark, RecoReward-9B outperforms its Qwen3.5-9B baseline and all other evaluated models across seven recall metrics. Online A/B testing also shows performance gains. These results show that RecoReward trains the MLLM to produce item features that benefit downstream recommendation while retaining content-only serving.

Problem

• multimodal recommendation
• description generation
• user behavior signals
• content-only inference
• recommender systems

Innovation

• RecoReward
• multimodal large language models
• recommender affinity score
• reinforcement learning
• content-only inference