From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models

2026-08-04

AI Summary

This study addresses the disconnect between financial sentiment classification accuracy and its predictive power for stock returns by establishing a unified three-class benchmark to systematically evaluate the sentiment recognition performance of various models and their out-of-sample forecasting efficacy for future returns. For the first time, QLoRA-based efficient fine-tuning is applied to Qwen2.5-7B, LLaMA3-8B, and Mistral-7B, with TF-IDF Naive Bayes, FinBERT, and Financial-RoBERTa serving as baselines. Inverse frequency-weighted loss and Newey-West corrected statistical inference are employed throughout. Experimental results show that Mistral-7B achieves a classification accuracy of 0.8840, and QLoRA substantially enhances Qwen2.5’s performance; however, none of the models generate statistically significant economic return signals, revealing the critical insight that high classification accuracy does not necessarily translate into tradable predictive power.

Abstract

Financial sentiment classifiers are commonly evaluated against human labels, but strong linguistic performance does not necessarily imply economically useful return predictability. This study separates these questions through two experiments. First, we construct a unified three-class benchmark from five financial text datasets and compare TF--IDF Naive Bayes, off-the-shelf FinBERT and Financial-RoBERTa encoders, zero-shot Qwen2.5-7B, and QLoRA-adapted Qwen2.5-7B, LLaMA3-8B, and Mistral-7B models. Mistral-7B achieves the best test accuracy (0.8840) and macro-F1 (0.8771), while QLoRA raises Qwen2.5's macro-F1 from 0.7274 to 0.8615. An inverse-frequency class-weighted loss does not improve Qwen2.5. Second, we evaluate economic validity on a temporally separate 2019 Benzinga sample containing 10,637 unique headlines and 13,115 headline--stock observations for a fixed S\&P~100 universe. Model probabilities are converted into continuous sentiment scores, aggregated by stock and signal date, and aligned with next-session returns over one-, two-, three-, and five-day horizons. All seven downstream models produce positive but small mean rank information coefficients at the one-day horizon; the largest is 0.0143 for FinBERT. None of the 28 model--horizon tests remains significant after Newey--West inference and false-discovery-rate correction. Portfolio results likewise fail to establish a robust advantage for the best-performing classifiers. The findings show that QLoRA is effective for financial sentiment adaptation, while also documenting a clear gap between classification accuracy and tradable cross-sectional signals.

Problem

• financial sentiment classification
• return predictability
• large language models
• economic validity
• cross-sectional signals

Innovation

• QLoRA
• financial sentiment classification
• return predictability
• large language models
• economic validity