Effective Dense Retrieval using Only In-Context Examples

2026-09-29

AI Summary

This study addresses the heavy reliance on fine-tuning when constructing dense retrievers from large language models (LLMs). To this end, it proposes RICE, a training-free retrieval-augmented method that leverages in-context learning. Through prompt engineering, RICE constructs a shared-context example set for each query, guiding the LLM to directly extract high-quality text representations without parameter updates, thereby departing from conventional fine-tuning paradigms. Experimental results demonstrate that RICE significantly improves embedding accuracy and downstream retrieval performance. This work establishes a new plug-and-play paradigm for deploying LLMs in retrieval tasks, and the corresponding code has been made publicly available.

Abstract

Turning decoder-only large language models (LLMs) into strong dense retrievers typically requires some form of retriever training. In this paper, we ask whether LLMs can instead be prompted to produce effective representations for dense retrieval given only a few in-context examples. To answer this, we introduce RICE (Representations from In-Context Examples), a simple "training-free" approach that extracts high-quality dense representations from LLMs. To do so, RICE conditions the LLM on examples that provide a shared context for query and document encoding. Our results demonstrate that RICE embeddings can substantially improve the accuracy of prompt-based LLM embeddings, establishing it as a simple method to build LLM-based dense retrievers that do not require training. We release our code at https://github.com/nourj98/RICE.

Problem

• Dense Retrieval
• Large Language Models
• In-Context Examples
• Training-free

Innovation

• Dense Retrieval
• In-Context Learning
• Training-free
• Large Language Models
• Representation Extraction