Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systems

2025-03-19

AI Summary

To address low retrieval accuracy and poor modeling of tabular structures in financial-domain RAG systems applied to complex documents (e.g., 10-K filings), this paper proposes a three-stage end-to-end optimization framework: pre-retrieval, retrieval, and post-retrieval. Methodologically, it innovatively integrates (1) financial-domain adaptive embedding fine-tuning, (2) hybrid dense-sparse retrieval, (3) DPO-based document re-ranking, and (4) multi-level table-structure-aware preprocessing. This tri-level collaborative paradigm significantly improves retrieval quality and factual consistency for financial question answering. The approach achieves state-of-the-art performance across seven financial QA benchmarks, including FinDER. All code is publicly released to ensure full reproducibility of the entire pipeline.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a promising framework to mitigate hallucinations in Large Language Models (LLMs), yet its overall performance is dependent on the underlying retrieval system. In the finance domain, documents such as 10-K reports pose distinct challenges due to domain-specific vocabulary and multi-hierarchical tabular data. In this work, we introduce an efficient, end-to-end RAG pipeline that enhances retrieval for financial documents through a three-phase approach: pre-retrieval, retrieval, and post-retrieval. In the pre-retrieval phase, various query and corpus preprocessing techniques are employed to enrich input data. During the retrieval phase, we fine-tuned state-of-the-art (SOTA) embedding models with domain-specific knowledge and implemented a hybrid retrieval strategy that combines dense and sparse representations. Finally, the post-retrieval phase leverages Direct Preference Optimization (DPO) training and document selection methods to further refine the results. Evaluations on seven financial question answering datasets-FinDER, FinQABench, FinanceBench, TATQA, FinQA, ConvFinQA, and MultiHiertt-demonstrate substantial improvements in retrieval performance, leading to more accurate and contextually appropriate generation. These findings highlight the critical role of tailored retrieval techniques in advancing the effectiveness of RAG systems for financial applications. A fully replicable pipeline is available on GitHub: https://github.com/seohyunwoo-0407/GAR.

Problem

• Enhances retrieval for financial documents in RAG systems
• Addresses domain-specific vocabulary and tabular data challenges
• Improves accuracy and contextuality in financial question answering

Innovation

• Three-phase RAG pipeline for financial documents
• Hybrid retrieval strategy with dense and sparse representations
• DPO training and document selection for result refinement