LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback

2025-04-02

AI Summary

This work investigates the generality and effectiveness of Vector Pseudo-Relevance Feedback (VPRF) in large language model (LLM)-driven dense retrieval. Addressing the limitation that existing VPRF methods are primarily designed for BERT-style models and poorly adaptable to LLM architectures, we propose LLM-VPRF—a novel paradigm that enhances queries via iterative optimization of query vectors in the embedding space. To our knowledge, this is the first systematic extension of VPRF to LLMs (e.g., Llama, Qwen). Our approach integrates LLM-based dense encoders, an iterative vector-space feedback mechanism, and cross-model robustness validation. We conduct comprehensive evaluation on BEIR and MSMARCO benchmarks. Experimental results demonstrate that LLM-VPRF consistently improves retrieval performance, yielding average gains of 3.2–5.7 percentage points in Recall@10 and MRR. It effectively bridges the methodological gap between BERT- and LLM-based pseudo-relevance feedback, confirming its cross-model efficacy and architectural transferability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Vector Pseudo Relevance Feedback (VPRF) has shown promising results in improving BERT-based dense retrieval systems through iterative refinement of query representations. This paper investigates the generalizability of VPRF to Large Language Model (LLM) based dense retrievers. We introduce LLM-VPRF and evaluate its effectiveness across multiple benchmark datasets, analyzing how different LLMs impact the feedback mechanism. Our results demonstrate that VPRF's benefits successfully extend to LLM architectures, establishing it as a robust technique for enhancing dense retrieval performance regardless of the underlying models. This work bridges the gap between VPRF with traditional BERT-based dense retrievers and modern LLMs, while providing insights into their future directions.

Problem

• Extends VPRF to LLM-based dense retrievers
• Evaluates LLM-VPRF on multiple benchmark datasets
• Bridges gap between VPRF and modern LLMs

Innovation

• Extends VPRF to LLM-based dense retrievers
• Evaluates LLM-VPRF across multiple datasets
• Analyzes LLM impact on feedback mechanism