TLOB: A Novel Transformer Model with Dual Attention for Stock Price Trend Prediction with Limit Order Book Data

2025-02-12

AI Summary

Short-term price trend prediction (SPTP) in limit order book (LOB)-driven markets suffers from poor generalization and unreliable short-horizon forecasts. Method: We propose a novel dual-attention Transformer architecture featuring: (i) a spatial-temporal dual-attention mechanism to jointly model the depth-wise structural properties and dynamic evolution of the LOB; (ii) an average-spread-driven trend labeling scheme that mitigates temporal horizon bias; and (iii) the first quantitative characterization of the time-decaying predictability of stock prices and the threshold effect of transaction costs on strategy viability. Contribution/Results: Evaluated on the FI-2010 benchmark and real-world Tesla and Intel LOB data, our model achieves F1-score improvements of 3.7%, 1.3%, and 7.7%, respectively. The implementation is publicly available.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Learning to Search
• Machine Learning: Time-Series/Data Streams

Application Category

• Economics, Online Markets and Human Computation: The sharing economy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Data transparency and provenance

Abstract

Stock Price Trend Prediction (SPTP) based on Limit Order Book (LOB) data is a fundamental challenge in financial markets. Despite advances in deep learning, existing models fail to generalize across different market conditions and struggle to reliably predict short-term trends. Surprisingly, by adapting a simple MLP-based architecture to LOB, we show that we surpass SoTA performance; thus, challenging the necessity of complex architectures. Unlike past work that shows robustness issues, we propose TLOB, a transformer-based model that uses a dual attention mechanism to capture spatial and temporal dependencies in LOB data. This allows it to adaptively focus on the market microstructure, making it particularly effective for longer-horizon predictions and volatile market conditions. We also introduce a new labeling method that improves on previous ones, removing the horizon bias. We evaluate TLOB's effectiveness using the established FI-2010 benchmark, which exceeds the state-of-the-art by an average of 3.7 F1-score(%). Additionally, TLOB shows improvements on Tesla and Intel with a 1.3 and 7.7 increase in F1-score(%), respectively. Additionally, we empirically show how stock price predictability has declined over time (-6.68 absolute points in F1-score(%)), highlighting the growing market efficiencies. Predictability must be considered in relation to transaction costs, so we experimented with defining trends using an average spread, reflecting the primary transaction cost. The resulting performance deterioration underscores the complexity of translating trend classification into profitable trading strategies. We argue that our work provides new insights into the evolving landscape of stock price trend prediction and sets a strong foundation for future advancements in financial AI. We release the code at https://github.com/LeonardoBerti00/TLOB.

Problem

• Predict stock trends using Limit Order Book data.
• Improve short-term trend prediction reliability.
• Adapt to volatile market conditions effectively.

Innovation

• Transformer model with dual attention
• Adaptive focus on market microstructure
• New labeling method reduces horizon bias