Alternative Loss Function in Evaluation of Transformer Models

2025-07-22

AI Summary

Conventional loss functions (e.g., MSE, MAE) are suboptimal for quantitative financial forecasting, as they prioritize absolute prediction accuracy over directional correctness—critical for algorithmic trading decisions. Method: We propose Mean Absolute Directional Loss (MADL), a decision-oriented loss function explicitly designed to optimize directional accuracy of trading signals. MADL is integrated into both Transformer and LSTM architectures and rigorously evaluated across multi-asset financial time series—including equities and cryptocurrencies—through systematic training, validation, hyperparameter tuning, and strategy backtesting. Contribution/Results: Empirical results demonstrate that MADL substantially enhances models’ directional discrimination capability in signal generation. Specifically, the MADL-optimized Transformer outperforms both LSTM and baseline models trained with conventional losses across key metrics: prediction accuracy, strategy profitability (e.g., Sharpe ratio, cumulative return), and robustness to market regime shifts. This work introduces a novel, decision-focused loss paradigm for financial time-series forecasting, bridging the gap between predictive modeling and real-world trading objectives.

Technology Category

Machine Learning: Quantum Machine LearningSearch and Optimization: Algorithm ConfigurationMultiagent Systems: Adversarial Agents

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

The proper design and architecture of testing of machine learning models, especially in their application to quantitative finance problems, is crucial. The most important in this process is selecting an adequate loss function used for training, validation, estimation purposes, and tuning of hyperparameters. Therefore, in this research, through empirical experiments on equity and cryptocurrency assets, we introduce the Mean Absolute Directional Loss (MADL) function which is more adequate for optimizing forecast-generating models used in algorithmic investment strategies. The MADL function results are compared for Transformer and LSTM models and we show that almost in every case Transformer results are significantly better than those obtained with LSTM.

Problem

• Introducing MADL for optimizing forecast models in finance
• Comparing Transformer and LSTM performance using MADL
• Selecting loss functions for ML models in algorithmic trading

Innovation

• Introducing Mean Absolute Directional Loss (MADL)
• Comparing Transformer and LSTM model performance
• Optimizing forecast models for algorithmic investment