Reward Guided Decoding for Generative Recommendation

2026-07-28

AI Summary

This work addresses a key limitation of generative recommender systems, which rely solely on generation probabilities during decoding and thus tend to overlook candidates with high business value but low likelihood, making them ill-suited for dynamic commercial objectives. To overcome this, the authors propose Reward-Guided Decoding (RGD), a framework that integrates reward signals with generation probabilities in closed form during decoding. By reshaping the search trajectory through KL-regularized reward maximization, RGD enables value-oriented, controllable recommendation without requiring model retraining. The approach combines autoregressive generative models with a test-time reward controller to optimize beam search. Extensive offline and online experiments on the Kuaishou platform demonstrate that RGD significantly enhances recommendation performance and consistently improves core business metrics.

Abstract

Generative recommendation formulates recommendation task into an SID sequence autoregressive generation paradigm, but the decoding process is often dominated by generation likelihood. This may conflict with real-world business objectives, where high-value candidates can receive low generation probability and be pruned early during beam search. Existing reranking or training-time alignment methods either intervene too late or require costly model retraining when business preferences change. To this end, we propose \textbf{R}eward \textbf{G}uided \textbf{D}ecoding, named \textbf{RGD}, a controllable decoding framework for industrial value-oriented generative recommendation. We formulate value-guided decoding as a KL-regularized reward maximization problem, deriving a closed-form reward guided decoding distribution that principledly combines generation probability with reward signals. RGD treats the base generator as a reference policy and introduces a reward model as a test-time controller, injecting reward into each decoding step to reshape the search trajectory without retraining the generator. Extensive offline and online experiments demonstrate the effectiveness of our approach for aligning personalization and business value. RGD has been deployed on the Kuaishou platform, bringing consistent improvements in real-world recommendation scenarios.

Problem

• generative recommendation
• reward-guided decoding
• business value alignment
• autoregressive generation
• beam search pruning

Innovation

• Reward Guided Decoding
• Generative Recommendation
• KL-Regularized Optimization
• Controllable Decoding
• Value Alignment