Enhancing LLMs through human feedback: a journey towards self-improvement

2026-07-13

AI Summary

This work proposes a human-feedback-driven dual-RAG architecture designed to continuously enhance the accuracy, relevance, and overall quality of Retrieval-Augmented Generation (RAG) systems through a human-in-the-loop mechanism. The approach introduces an auxiliary feedback RAG module that automatically collects, categorizes, and integrates user feedback into the primary RAG inference pipeline, enabling autonomous iterative refinement without requiring explicit supervisory signals. Leveraging an LLM-as-a-Judge evaluation strategy, the method demonstrates significant improvements in response quality across three benchmark datasets encompassing both general and domain-specific knowledge, thereby advancing RAG systems toward self-optimizing capabilities.

Abstract

In the rapidly evolving landscape of information retrieval systems, the ability to adapt and improve through user feedback is paramount. This study introduces a novel methodology for refining the performance of a primary Retrieval Augmented Generation (RAG) system by strategically integrating an auxiliary feedback RAG system. By systematically harnessing human-generated feedback, the approach aims to enhance the accuracy, relevance, and overall quality of responses, driving the system towards self-improvement. Central to this methodology is a human-in-the-loop implementation, where user feedback is continuously collected, classified, and integrated into the inference workflow, enabling the system to learn and evolve iteratively. To validate the effectiveness of this approach, the study employs rigorous testing against three diverse benchmark datasets focused on general and custom domain knowledge, utilizing a LLM-as-a-Judge evaluation strategy. This comprehensive framework not only underscores the transformative potential of feedback-driven enhancements in RAG systems but also sets a precedent for future research in adaptive information retrieval technologies, marking a significant step in the journey towards autonomous refinement and optimization through user engagement.

Problem

• human feedback
• Retrieval Augmented Generation
• self-improvement
• information retrieval
• LLM

Innovation

• Retrieval Augmented Generation
• human-in-the-loop
• self-improvement
• feedback integration
• LLM-as-a-Judge