Nek Minit: Harnessing Pragmatic Metacognitive Prompting for Explainable Sarcasm Detection of Australian and Indian English

2025-05-21

AI Summary

This work addresses the explainability challenge in geographically grounded sarcasm detection for Australian and Indian English—specifically, the inconsistency between surface sentiment and implicit sentiment, and its dependence on regional linguistic conventions. We propose Practical Metacognitive Prompting (PMP), the first method to generate interpretable explanations for sarcasm across English language varieties. To support evaluation, we introduce BESSTIE, the first manually annotated, cross-regional benchmark dataset containing Australian and Indian English samples. Experiments with open-source LLMs (e.g., Gemma, LLaMA) show that PMP significantly outperforms four baseline prompting strategies in both sarcasm detection accuracy and explanation quality; agent-style prompting further mitigates context-length degradation. Our key contributions are: (1) a PMP paradigm explicitly modeling region-dependent implicit sentiment; (2) the release of BESSTIE—the first explainable, cross-regional sarcasm detection benchmark; and (3) empirical validation that PMP simultaneously enhances model generalizability and interpretability.

Technology Category

• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Search and Optimization: Metareasoning and Metaheuristics
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Sarcasm is a challenge to sentiment analysis because of the incongruity between stated and implied sentiment. The challenge is exacerbated when the implication may be relevant to a specific country or geographical region. Pragmatic metacognitive prompting (PMP) is a cognition-inspired technique that has been used for pragmatic reasoning. In this paper, we harness PMP for explainable sarcasm detection for Australian and Indian English, alongside a benchmark dataset for standard English. We manually add sarcasm explanations to an existing sarcasm-labeled dataset for Australian and Indian English called BESSTIE, and compare the performance for explainable sarcasm detection for them with FLUTE, a standard English dataset containing sarcasm explanations. Our approach utilising PMP when evaluated on two open-weight LLMs (GEMMA and LLAMA) achieves statistically significant performance improvement across all tasks and datasets when compared with four alternative prompting strategies. We also find that alternative techniques such as agentic prompting mitigate context-related failures by enabling external knowledge retrieval. The focused contribution of our work is utilising PMP in generating sarcasm explanations for varieties of English.

Problem

• Detecting sarcasm in Australian and Indian English
• Explaining sarcasm using metacognitive prompting
• Improving performance with pragmatic reasoning techniques

Innovation

• Uses Pragmatic Metacognitive Prompting (PMP) technique
• Applies PMP for explainable sarcasm detection
• Evaluates on GEMMA and LLAMA LLMs