Parameters vs. Context: TRACE Fine-Tuning for Robust Retrieval-Augmented Generation

2026-09-24

AI Summary

This study addresses the challenge in Retrieval-Augmented Generation (RAG) where conflicts between externally retrieved knowledge and internal parametric knowledge often lead to unreliable responses. To mitigate this issue, this work proposes TRACE, a novel framework that introduces a fine-grained supervision mechanism for knowledge source selection based on multi-agent debate trajectories. Furthermore, it incorporates answer completeness regularization during model fine-tuning. By mining high-quality supervision signals from these debate trajectories and reinforcing the generation of tail segments in answers, the proposed approach significantly enhances the model's robustness against misleading knowledge while effectively alleviating the problem of incomplete responses.

Abstract

Retrieval-Augmented Generation (RAG) mitigates knowledge obsolescence and factual hallucination in large language models by introducing external context. However, when retrieved knowledge conflicts with the model's internal parametric knowledge, the model may either blindly follow misleading context or incorrectly rely on parametric knowledge, leading to unreliable responses. To address this issue, this paper proposes TRACE (Debate-TRace and Answer-Completeness rEgularized fine-tuning), a robust fine-tuning framework for RAG under knowledge conflicts. First, we propose a fine-tuning method that leverages multi-agent debate traces to extract correct candidates, incorrect candidates, and answer-shift patterns, providing fine-grained supervision for reliable knowledge-source selection. In addition, we design an answer completeness regularization mechanism to alleviate empty, overly short, and prematurely terminated responses via answer-tail token reinforcement and premature termination suppression. The fine-tuning objective combines correct-answer supervision, incorrect-candidate suppression, answer-tail token reinforcement, and premature termination suppression, enabling the model to use reliable external context, resist misleading or irrelevant retrieved content, and fall back to parametric knowledge when retrieved evidence is unreliable. Experiments across multiple knowledge-conflict scenarios and datasets show that TRACE improves robustness against misleading retrieved knowledge and reduces incomplete answers. These results demonstrate that multi-agent debate traces and answer completeness regularization jointly enhance knowledge-source selection, conflict robustness, and answer quality in RAG models. Our code is available at https://github.com/PHD-lanyu/TRACE.

Problem

• Retrieval-Augmented Generation
• Knowledge Conflict
• Parametric Knowledge
• Robustness
• Answer Completeness

Innovation

• Retrieval-Augmented Generation
• Multi-Agent Debate
• Knowledge Conflict
• Fine-Tuning
• Answer Completeness Regularization