ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval

2025-10-11

AI Summary

Generative retrieval (GR) excels in supervised settings but suffers from poor generalization in zero-shot information retrieval (IR). This paper introduces ZeroGR—the first general-purpose generative framework for zero-shot retrieval over heterogeneous documents (e.g., text, tables, code). To address the challenge of format-agnostic semantic alignment, ZeroGR employs: (1) an instruction-tuned query generator that unifies modeling across diverse document modalities; and (2) a reverse-annealing decoding strategy to enhance generation stability and relevance. The framework is end-to-end optimized and seamlessly integrates large language models. Evaluated on the BEIR and MAIR benchmarks under strict zero-shot settings, ZeroGR significantly outperforms both dense retrievers and existing generative baselines, achieving state-of-the-art performance. Results demonstrate ZeroGR’s strong cross-task generalizability, scalability to unseen tasks, and robustness across document formats—without any task-specific fine-tuning.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Generative retrieval (GR) reformulates information retrieval (IR) by framing it as the generation of document identifiers (docids), thereby enabling an end-to-end optimization and seamless integration with generative language models (LMs). Despite notable progress under supervised training, GR still struggles to generalize to zero-shot IR scenarios, which are prevalent in real-world applications. To tackle this challenge, we propose extsc{ZeroGR}, a zero-shot generative retrieval framework that leverages natural language instructions to extend GR across a wide range of IR tasks. Specifically, extsc{ZeroGR} is composed of three key components: (i) an LM-based docid generator that unifies heterogeneous documents (e.g., text, tables, code) into semantically meaningful docids; (ii) an instruction-tuned query generator that generates diverse types of queries from natural language task descriptions to enhance corpus indexing; and (iii) a reverse annealing decoding strategy to balance precision and recall during docid generation. We investigate the impact of instruction fine-tuning scale and find that performance consistently improves as the number of IR tasks encountered during training increases. Empirical results on the BEIR and MAIR benchmarks demonstrate that extsc{ZeroGR} outperforms strong dense retrieval and generative baselines in zero-shot settings, establishing a new state-of-the-art for instruction-driven GR.

Problem

• Extends generative retrieval to zero-shot scenarios using instructions
• Unifies heterogeneous documents into semantic identifiers for retrieval
• Balances precision and recall in document ID generation

Innovation

• Generative retrieval framework using natural language instructions
• Unifies heterogeneous documents into semantic document identifiers
• Instruction-tuned query generator with reverse annealing decoding