Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation

2026-06-02

AI Summary

This work addresses key challenges in large language model (LLM)-enhanced recommender systems, including misalignment between semantic and ID embedding spaces, difficulty in evaluating chain-of-thought (CoT) quality, and the lack of principled multi-objective reward balancing. To tackle these issues, the authors propose the Taiji framework, which constructs high-quality, domain-specific CoT data through backward reasoning and open-ended rejection sampling, and introduces a Pareto-Optimal Policy Optimization (POPO) algorithm to adaptively balance semantic understanding and user preference rewards during reinforcement learning. Notably, this is the first approach to jointly optimize LLM semantic capabilities and ID-space features in an industrial-scale recommender system. Deployed on Kuaishou’s advertising platform serving over 400 million daily users, Taiji demonstrates significant gains in commercial revenue, with both offline and online experiments confirming its effectiveness and scalability.

Abstract

Scaling recommender systems via large language models (LLMs) has become a prominent trend in the industry. However, aligning the LLM's semantic space with the recommender's ID space via post-training (e.g., SFT and RL) remains challenging. Existing LLM4Rec paradigms are bottlenecked by two main issues: (1) the difficulty of measuring and improving chain-of-thought (CoT) quality in open-domain recommendation during SFT, and (2) the neglect of the trade-off between LLM semantic rewards and recommendation preference rewards during RL alignment. Inspired by these challenges, we present Taiji, a novel LLM-as-Enhancer framework designed for industrial recommender systems. To overcome the SFT bottleneck, we utilize reverse-engineered reasoning and open-ended rejection sampling to generate high-quality, domain-specific CoT data. To resolve the RL alignment issue, we propose Pareto Optimal Policy Optimization (POPO), which adaptively adjusts cross-domain reward weights. Theoretically, it achieves an optimal trade-off between the semantic world knowledge of LLMs and the collaborative ID features representing online user preferences. Extensive offline evaluations and online A/B tests validate the effectiveness of Taiji. Deployed on Kuaishou's advertising platform since May 2026, Taiji currently serves over 400 million users daily, yielding significant commercial revenue and demonstrating its robust scalability in web-scale environments.

Problem

• LLM-enhanced recommendation
• semantic-ID alignment
• chain-of-thought quality
• reward trade-off
• Pareto optimality

Innovation

• Pareto Optimal Policy Optimization
• Chain-of-Thought
• LLM-enhanced Recommendation
• Semantic-ID Alignment
• Reverse-engineered Reasoning