Enhanced Retrieval of Long Documents: Leveraging Fine-Grained Block Representations with Large Language Models

2025-01-28

AI Summary

To address the limitation of coarse-grained embeddings in modeling fine-grained semantic details for long-document retrieval, this paper proposes a fine-grained chunked embedding and weighted relevance aggregation framework. First, documents are segmented into semantically coherent paragraph-level chunks, and large language models generate chunk-level embeddings. Second, chunk–query relevance scores are computed and aggregated via a learnable weighting mechanism to produce precise document-level representations. Third, an optimized pairwise ranking loss is introduced to enhance retrieval ranking performance. Evaluated on multiple long-document retrieval benchmarks, the method significantly outperforms single-document embedding baselines, achieving substantial gains in retrieval accuracy while reducing embedding generation latency by over 40%. The approach thus achieves a favorable trade-off between efficiency and effectiveness.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In recent years, large language models (LLMs) have demonstrated exceptional power in various domains, including information retrieval. Most of the previous practices involve leveraging these models to create a single embedding for each query, each passage, or each document individually, a strategy exemplified and used by the Retrieval-Augmented Generation (RAG) framework. While this method has proven effective, we argue that it falls short in fully capturing the nuanced intricacies of document-level texts due to its reliance on a relatively coarse-grained representation. To address this limitation, we introduce a novel, fine-grained approach aimed at enhancing the accuracy of relevance scoring for long documents. Our methodology firstly segments a long document into blocks, each of which is embedded using an LLM, for matching with the query representation. When calculating the relevance score, we aggregate the query-block relevance scores through a weighted sum method, yielding a comprehensive score for the query with the entire document. Despite its apparent simplicity, our experimental findings reveal that this approach outperforms standard representation methods and achieves a significant reduction in embedding generation latency. Moreover, by carefully optimizing pairwise loss functions, superior performances have been achieved.

Problem

• Long Document Retrieval
• Detail Capturing
• Complex Information Processing

Innovation

• Large Language Models
• Text Chunking
• Weighted Similarity Scoring