🤖 AI Summary
This work addresses retrieval bias and spurious evidence in SEC filing question answering, which arise from mismatches between model priors and the documents’ structure, terminology, and evidentiary standards. To mitigate these issues, the authors propose a role-specialized multi-agent framework that aligns query decomposition with 10-K filing structures, employs corpus-aware retrieval planning, and incorporates a multi-path evidence verification mechanism for precise answers. A key innovation is the decoupling of evidence validity from semantic similarity, achieved by injecting corpus-side conditions during query generation and reranking, complemented by a lightweight local corpus overview to guide agent collaboration. Experiments demonstrate that the approach significantly improves retrieval coverage and answer accuracy across five financial QA benchmarks and outperforms existing baselines in a large-scale online evaluation involving over one thousand participants.
📝 Abstract
Financial question answering over U.S. Securities and Exchange Commission (SEC) filings requires retrieving and synthesizing heterogeneous evidence dispersed across long, standardized, and highly redundant disclosures. Existing retrieval-augmented and multi-agent systems typically derive retrieval queries directly from the user's question and rank candidates by semantic similarity. Together, these choices create prior-corpus misalignment: a mismatch between model priors and the target filings' structure, terminology, and evidence standards. As a result, query generation misses corpus-specific evidence, while semantic reranking favors topically similar but evidentially invalid false-positive chunks. We propose FinSAgent, an evidence-grounded multi-agent framework that reframes SEC filing QA as corpus-aligned retrieval planning and corrects both ends with a single principle: inject corpus-side conditioning wherever model priors would otherwise dominate. FinSAgent combines (1) role-specialized agents anchored to the mandated 10-K item structure, (2) database-aware query decomposition that conditions each agent's sub-queries on a lightweight, summary-level view of the local corpus, and (3) multi-path retrieval with a learned feature-gated reranker that separates evidential validity from semantic similarity. Across five offline financial QA benchmarks, FinSAgent improves retrieval coverage and answer correctness over strong single-agent and multi-agent baselines; in a three-arm randomized online experiment with 1,000 anonymous user ratings, it also receives higher scores than baselines.