Seek: Self-Evaluative Exploration for Knowledge Retrieval

2026-09-23

AI Summary

This study addresses the irrecoverability of missed documents in single-pass retrieval by proposing a training-free iterative retrieval framework. The method introduces a test-time self-evaluation exploration mechanism that constructs a feedback loop through large language model-generated pseudo-passages, iterative retrieval of new candidates, and hierarchical relevance assessment, thereby overcoming the limitations of conventional single-pass interaction. Evaluated on the BRIGHT benchmark, the proposed approach achieves an 82% improvement over BM25 and outperforms all training-required baselines, demonstrating a significant advancement in retrieval performance without the need for model training.

Abstract

LLM-based retrievers and rerankers have advanced passage ranking, yet both paradigms interact with the corpus in a single pass and commit to the resulting candidate set, leaving relevant documents permanently unrecoverable once missed. We introduce Seek, Self-Evaluative Exploration for Knowledge Retrieval, a training-free framework that addresses this limitation through iterative corpus interaction at test time. At each round, an LLM generates pseudo-passages conditioned on accumulated relevance feedback, a retriever surfaces fresh candidates, and a dedicated assessor assigns graded relevance judgments that guide subsequent rounds. On TREC Deep Learning, Seek matches trained rerankers in ranking quality while consistently improving Recall@100 over single-pass BM25. On the reasoning-intensive BRIGHT benchmark, Seek with Qwen2.5-7B achieves an 82% relative gain over BM25, surpassing all trained baselines, and Seek with GPT-4.1 reaches 37.4 average nDCG@10, exceeding the strongest baseline by 37%.

Problem

• Knowledge Retrieval
• Passage Ranking
• Single-pass Retrieval
• Recall
• Large Language Models

Innovation

• Self-Evaluative Exploration
• Training-free Framework
• Iterative Retrieval
• Pseudo-passage Generation
• Knowledge Retrieval