Reasoning to Rank: An End-to-End Solution for Exploiting Large Language Models for Recommendation

2026-02-13

Technology Category

Search and Optimization: Learning to SearchMachine Learning: Reinforcement LearningData Mining & Knowledge Management: Recommender Systems

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recommender systems are tasked to infer users'evolving preferences and rank items aligned with their intents, which calls for in-depth reasoning beyond pattern-based scoring. Recent efforts start to leverage large language models (LLMs) for recommendation, but how to effectively optimize the model for improved recommendation utility is still under explored. In this work, we propose Reasoning to Rank, an end-to-end training framework that internalizes recommendation utility optimization into the learning of step-by-step reasoning in LLMs. To avoid position bias in LLM reasoning and enable direct optimization of the reasoning process, our framework performs reasoning at the user-item level and employs reinforcement learning for end-to-end training of the LLM. Experiments on three Amazon datasets and a large-scale industrial dataset showed consistent gains over strong conventional and LLM-based solutions. Extensive in-depth analyses validate the necessity of the key components in the proposed framework and shed lights on the future developments of this line of work.

Problem

• recommender systems
• large language models
• reasoning
• ranking
• recommendation utility

Innovation

• Reasoning to Rank
• Large Language Models
• Reinforcement Learning
• End-to-End Training
• Recommendation Systems