Intermediate-Task Transfer Learning: Leveraging Sarcasm Detection for Stance Detection

2025-03-05

AI Summary

This work addresses the degradation in stance detection performance on social media caused by sarcastic and metaphorical language. To mitigate this issue, we propose a transfer learning framework that incorporates irony detection as an intermediate task. Methodologically, we introduce irony recognition as an auxiliary task within the stance detection pipeline for the first time, and design a novel hybrid architecture integrating BERT/RoBERTa with a convolutional BiLSTM–fully connected layer to jointly model deep semantics and local linguistic patterns. Experiments across multiple public benchmarks demonstrate substantial improvements over existing state-of-the-art models, with significant average F1-score gains; notably, the framework corrects 85% of originally misclassified instances, confirming its robustness against rhetorical interference. Our core contributions are: (1) the first irony-detection-driven transfer paradigm for stance detection, and (2) a hybrid neural architecture that synergistically combines contextualized semantic representation with sequential pattern modeling.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Stance Detection (SD) on social media has emerged as a prominent area of interest with implications for social business and political applications thereby garnering escalating research attention within NLP. The inherent subtlety and complexity of texts procured from online platforms pose challenges for SD algorithms in accurately discerning the authors stance. Mostly the inclusion of sarcastic and figurative language drastically impacts the performance of SD models. This paper addresses this by employing sarcasm detection intermediate-task transfer learning tailored for SD. The proposed methodology involves the finetuning of BERT and RoBERTa and the concatenation of convolutional BiLSTM and dense layers. Rigorous experiments are conducted on publicly available datasets to evaluate our transfer-learning framework. The performance of the approach is assessed against various State-Of-The-Art baselines for SD providing empirical evidence of its effectiveness. Notably our model outperforms the best SOTA models even prior to sarcasm-detection pretraining. The integration of sarcasm knowledge into the model proves instrumental in mitigating misclassifications of sarcastic textual elements in SD. Our model accurately predicts 85% of texts that were previously misclassified by the model without sarcasm-detection pretraining thereby amplifying the average F1-score of the model. Our experiments also revealed that the success of the transfer-learning framework is contingent upon the correlation of lexical attributes between the intermediate task and the target task. This study represents the first exploration of sarcasm detection as an intermediate transfer-learning task in the context of SD and simultaneously uses the concatenation of BERT or RoBERTa with other deep-learning techniques establishing the proposed approach as a foundational baseline for future research endeavors in this domain.

Problem

• Improving stance detection accuracy on social media texts.
• Addressing sarcasm and figurative language challenges in stance detection.
• Exploring sarcasm detection as an intermediate transfer-learning task.

Innovation

• Uses sarcasm detection for stance detection
• Combines BERT, RoBERTa with convolutional BiLSTM
• Improves F1-score by mitigating sarcasm misclassifications