FinSphere: A Conversational Stock Analysis Agent Equipped with Quantitative Tools based on Real-Time Database

2025-01-08

arXiv.org

AI Summary

Existing financial large language models (FinLLMs) face two critical bottlenecks in stock analysis: the absence of objective, quantitative evaluation metrics for report quality and insufficient analytical depth to generate professional-grade insights. To address these limitations, we propose the first end-to-end conversational AI agent framework specifically designed for stock analysis. Our method integrates a real-time financial database, a quantitative computation module, and an instruction-tuned LLM to enable multi-step reasoning and structured output generation. Key contributions include: (1) the Stocksis dataset—curated and expert-annotated by finance professionals; (2) AnalyScore—a novel, interpretable, multi-dimensional evaluation metric for analytical report quality; and (3) a modular agent architecture supporting domain-specific reasoning. Experimental results demonstrate that our agent significantly outperforms both general-purpose and financial-domain LLMs, as well as existing agent systems, on professional stock analysis tasks—yielding substantial improvements in report quality, interpretability, and operational applicability.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent Communication
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Current financial Large Language Models (LLMs) struggle with two critical limitations: a lack of depth in stock analysis, which impedes their ability to generate professional-grade insights, and the absence of objective evaluation metrics to assess the quality of stock analysis reports. To address these challenges, this paper introduces FinSphere, a conversational stock analysis agent, along with three major contributions: (1) Stocksis, a dataset curated by industry experts to enhance LLMs' stock analysis capabilities, (2) AnalyScore, a systematic evaluation framework for assessing stock analysis quality, and (3) FinSphere, an AI agent that can generate high-quality stock analysis reports in response to user queries. Experiments demonstrate that FinSphere achieves superior performance compared to both general and domain-specific LLMs, as well as existing agent-based systems, even when they are enhanced with real-time data access and few-shot guidance. The integrated framework, which combines real-time data feeds, quantitative tools, and an instruction-tuned LLM, yields substantial improvements in both analytical quality and practical applicability for real-world stock analysis.

Problem

• Lack of objective metrics for stock analysis quality evaluation
• Insufficient depth in stock analysis by FinLLMs
• Need for professional-grade stock insights generation

Innovation

• Instruction-tuned LLMs for real-time stock analysis
• AnalyScore framework for evaluating analysis quality
• Integrated real-time data and quantitative tools