StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices

2026-01-01

arXiv.org

AI Summary

Financial time series forecasting remains highly challenging due to high volatility, nonlinear dynamics, and latent temporal dependencies. This study systematically evaluates the performance of vanilla LSTM, Transformer variants, and convolutional time series models under a unified experimental framework with default hyperparameters. The results demonstrate that a properly configured vanilla LSTM consistently outperforms attention-based models in both predictive accuracy and trading decision stability, exhibiting superior robustness and data efficiency. These findings highlight the significant advantage of the inductive bias inherent in recurrent architectures for stock market prediction tasks, particularly under data-scarce conditions and without extensive hyperparameter tuning, thereby challenging the prevailing paradigm that favors Transformer-based approaches.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Accurate forecasting of financial markets remains a long-standing challenge due to complex temporal and often latent dependencies, non-linear dynamics, and high volatility. Building on our earlier recurrent neural network framework, we present an enhanced StockBot architecture that systematically evaluates modern attention-based, convolutional, and recurrent time-series forecasting models within a unified experimental setting. While attention-based and transformer-inspired models offer increased modeling flexibility, extensive empirical evaluation reveals that a carefully constructed vanilla LSTM consistently achieves superior predictive accuracy and more stable buy/sell decision-making when trained under a common set of default hyperparameters. These results highlight the robustness and data efficiency of recurrent sequence models for financial time-series forecasting, particularly in the absence of extensive hyperparameter tuning or the availability of sufficient data when discretized to single-day intervals. Additionally, these results underscore the importance of architectural inductive bias in data-limited market prediction tasks.

Problem

• financial time-series forecasting
• stock price prediction
• market volatility
• temporal dependencies
• non-linear dynamics

Innovation

• vanilla LSTM
• financial time-series forecasting
• inductive bias
• data efficiency
• model comparison