Problem
Research questions and friction points this paper is trying to address.
financial indicator construction
Deep Research agents
benchmarking
end-to-end evaluation
financial data retrieval
FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Construction
🤖 AI Summary
This work addresses a critical gap in existing financial evaluation benchmarks, which predominantly focus on final answer accuracy while neglecting systematic assessment of the intermediate steps involved in financial indicator construction. The paper introduces the first end-to-end benchmark specifically designed for evaluating deep research agents in financial indicator synthesis, encompassing four stages: formula definition, data collection, indicator computation, and answer generation. Covering 21 subcategories across fundamental, technical, and macroeconomic domains, the benchmark leverages ten years of historical data and information from 800 listed companies. Systematic evaluation using retrieval-augmented large language models and deep research agents reveals that while large language models perform well in formula definition, their accuracy drops significantly during data retrieval and numerical computation. Although deep research agents demonstrate overall superiority, they still fall short of the reliability required for real-world financial analysis.
📝 Abstract
Financial indicators are essential tools for transforming raw financial data into interpretable measures for various downstream tasks, such as valuation, risk assessment, and economic analysis. However, existing financial benchmarks largely focus on answer-level accuracy and often assume that relevant data are already provided, leaving the assessment of the intermediate process of indicator construction underexplored. In this work, we propose FinDeepIndicator, the first benchmark dedicated to evaluating Deep Research (DR) agents in end-to-end financial indicator construction. Specifically, FinDeepIndicator evaluates DR agents across four stages in indicator construction: formula specification, data collection, indicator calculation, and answer generation, and covers fundamental, technical, and macroeconomic indicators organized into 21 fine-grained sub-categories. It contains 3,350 curated question-answer (QA) pairs derived from both U.S. and Chinese markets, 10 years of historical financial data, and 800 listed companies. Extensive experiments on search-equipped Large Language Models (LLMs) and DR agents show that, while LLMs generally perform well in formula specification, their accuracy drops substantially during data retrieval and numerical execution. DR agents consistently outperform search-equipped LLMs, yet remain unreliable in realistic financial analysis settings. These findings provide insights for developing more capable and trustworthy DR agents in finance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Deep Research Agents
Financial Indicator Construction
End-to-End Benchmarking
Large Language Models
Financial Reasoning
🔎 Similar Papers
No similar papers found.
