REARANK: Reasoning Re-ranking Agent via Reinforcement Learning

2025-05-26

AI Summary

Listwise reranking in information retrieval suffers from limited performance, poor interpretability, and heavy reliance on large-scale annotated data. Method: This paper proposes REARANK-7B, a reasoning-prioritized reranking agent built upon Qwen2.5-7B, integrating reinforcement learning optimization and lightweight data augmentation to achieve strong generalization with only 179 labeled samples. Its core innovation is the “reasoning-prior” paradigm—explicitly modeling list-level semantic relationships and decision logic to jointly enhance ranking accuracy and process interpretability. Results: REARANK-7B significantly outperforms conventional baselines across mainstream IR benchmarks. It matches GPT-4’s performance on in-domain and out-of-domain tasks, and—critically—achieves the first reported superiority over GPT-4 on the inference-intensive BRIGHT benchmark, demonstrating both efficiency and robustness in complex, reasoning-heavy retrieval scenarios.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present REARANK, a large language model (LLM)-based listwise reasoning reranking agent. REARANK explicitly reasons before reranking, significantly improving both performance and interpretability. Leveraging reinforcement learning and data augmentation, REARANK achieves substantial improvements over baseline models across popular information retrieval benchmarks, notably requiring only 179 annotated samples. Built on top of Qwen2.5-7B, our REARANK-7B demonstrates performance comparable to GPT-4 on both in-domain and out-of-domain benchmarks and even surpasses GPT-4 on reasoning-intensive BRIGHT benchmarks. These results underscore the effectiveness of our approach and highlight how reinforcement learning can enhance LLM reasoning capabilities in reranking.

Problem

• Improves ranking performance via reasoning-enhanced reranking
• Enhances interpretability in listwise ranking with LLMs
• Achieves GPT-4 level results with minimal annotated data

Innovation

• LLM-based listwise reasoning reranking agent
• Reinforcement learning enhances reasoning capabilities
• Minimal annotated samples for substantial improvements