Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation

2024-12-11

arXiv.org

AI Summary

In retrieval-augmented generation (RAG), a semantic gap exists between retriever rerankers and generative models in assessing document relevance. To bridge this gap, we propose RADIO, the first framework that jointly leverages large language model (LLM)-generated reasoning rationales and preference-aligned fine-tuning. Specifically, an LLM first extracts step-by-step reasoning traces required to answer the query; these rationales serve as supervision signals to rerank retrieved documents; finally, the reranker is further optimized via preference learning to align with generative behavior. This enables semantic-level coordination between reranking and generation. Experiments across three benchmark datasets and two task categories demonstrate that RADIO significantly improves answer quality and factual consistency over state-of-the-art baselines. The implementation is publicly available.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Machine Learning: Learning Preferences or Rankings
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The reranker and generator are two critical components in the Retrieval-Augmented Generation (i.e., RAG) pipeline, responsible for ranking relevant documents and generating responses. However, due to differences in pre-training data and objectives, there is an inevitable gap between the documents ranked as relevant by the reranker and those required by the generator to support answering the query. To address this gap, we propose RADIO, a novel and practical preference alignment framework with RAtionale DIstillatiOn. Specifically, We first propose a rationale extraction method that leverages the reasoning capabilities of Large Language Models (LLMs) to extract the rationales necessary for answering the query. Subsequently, a rationale-based alignment process is designed to rerank the documents based on the extracted rationales, and fine-tune the reranker to align the preferences. We conduct extensive experiments on two tasks across three datasets to demonstrate the effectiveness of our approach compared to baseline methods. Our code is released online to ease reproduction.

Problem

• Bridging gap between reranker and generator in RAG
• Aligning document relevance with generator reasoning needs
• Improving retrieval-augmented generation via rationale distillation

Innovation

• Leverages LLMs for rationale extraction
• Aligns reranker preferences via rationale distillation
• Fine-tunes reranker using rationale-based document ranking