Score
The ability to detect, quantify, and track sentiment and speech-act patterns in text or dialogue, distinguishing tones (e.g., mockery vs. gatekeeping) and their evolution. Used to identify social scaffolding, measure negative experiences, and enable compression or redaction that preserves semantic intent and affective meaning.
This study addresses the underexplored yet critical role of speech in irony detection, where prior work over-relies on textual cues and neglects cross-cultural and multilingual adaptability. We present the first dedicated survey on speech-centric irony recognition, systematically reviewing the evolution from unimodal speech to speech-text multimodal fusion. Our analysis covers benchmark datasets, acoustic feature engineering, deep representation learning, and multimodal integration techniques—highlighting the decisive influence of prosodic cues (e.g., intonation, rhythm) on ironic intent inference. Key contributions include: (1) establishing speech as the core modality reflecting irony’s multimodal nature; (2) identifying structural gaps in existing datasets regarding cultural coverage and linguistic diversity; and (3) proposing three future directions—cross-lingual acoustic modeling, culture-aware prosody analysis, and lightweight multimodal architectures—to advance irony computation from monolingual text-based paradigms toward universal, multimodal human-computer interaction.
This study addresses the lack of objective and interpretable vocal biomarkers in mental health assessment by proposing a transparent, clinically interpretable speech analysis framework. It systematically integrates multidimensional perceptual features—including prosody, voice quality, semantic coherence, syntactic structure, and sarcasm—by jointly leveraging acoustic and linguistic information. Using an XGBoost model enhanced with SHAP and LIME for interpretability, the framework identifies key features such as jitter, shimmer, lexical-syntactic patterns, and affective intonation from real-world clinical data and multiple benchmark datasets. Experimental results demonstrate robust associations between vocal irregularities and symptom severity in depression, anxiety, and ADHD. Ablation studies further confirm the most discriminative feature subsets, offering reliable and explainable vocal biomarkers to support clinical evaluation.
This study addresses the substantial inter-annotator disagreement and low stability observed in sentiment annotation. Grounded in Appraisal Theory, it systematically investigates the textual features of sentiment expression and their computability. Using a manually annotated industrial corpus, we construct a fine-grained sentiment dataset and train language models to emulate human annotation behavior. Methodologically, we move beyond mainstream sentiment classification paradigms by incorporating appraisal-oriented semantic structures—namely, Attitude, Engagement, and Graduation—which remain underexplored in computational linguistics. This enables us to uncover stable statistical patterns and language-driven mechanisms underlying annotation discrepancies. Experiments demonstrate that our model effectively discriminates among distinct appraisal-based sentiment contexts, achieving significant improvements over baselines in cross-context generalization and fine-grained linguistic cue modeling. Our work establishes a theory-informed modeling framework for sentiment computation and provides an interpretable, appraisal-grounded evaluation benchmark.
Modeling the dynamic evolution of emotional states in mental health texts remains challenging, particularly for capturing intra-message, sentence-level temporal shifts in emotion intensity (e.g., escalation or relief). Method: This paper introduces and formally defines “emotion drift”—a novel metric quantifying fine-grained, sequential changes in emotion intensity across sentences within a single message. Unlike conventional coarse-grained document-level sentiment classification, our approach jointly models sentence-level emotion recognition and drift scoring using pre-trained Transformer architectures (DistilBERT, RoBERTa). Contribution/Results: Experiments demonstrate that emotion drift effectively captures critical affective turning points in psychotherapeutic dialogues, significantly improving both interpretability and quantitative precision in early crisis detection and optimal intervention timing. The metric provides a clinically actionable, explainable signal, establishing a new paradigm for AI-augmented clinical decision support in mental healthcare.
Existing speech emotion recognition (SER) models are predominantly trained and evaluated on acted speech (e.g., podcasts), leading to emotional expression distortion; moreover, crowdsourced annotations often lack transparent, standardized guidelines, hindering model interpretability and targeted improvement. Method: Leveraging the Switchboard corpus of natural conversational speech, we construct SWB-Affect—the first systematically curated, highly transparent affective annotation set for spontaneous dialogue—covering 10 discrete emotions and three continuous affective dimensions (valence, arousal, dominance). We introduce explicit, reproducible annotation protocols integrating categorical and dimensional frameworks, and quantitatively analyze linguistic and paralinguistic cues’ contributions to emotion perception. Contribution/Results: We publicly release SWB-Affect. Experiments reveal substantial performance degradation of state-of-the-art SER models on natural speech—particularly for anger—validating the critical importance of real-world conversational data for robustness evaluation.
This study addresses the challenge of evaluating the effectiveness of counter-speech in mitigating online hate speech. We propose the first sociologically grounded, six-dimensional evaluation framework—encompassing clarity, evidentiality, emotional appeal, and other theoretically motivated dimensions—and conduct collaborative human annotation of 4,214 counter-speech instances, resulting in the first large-scale, structured counter-speech dataset, publicly released. Methodologically, we introduce a novel multi-task learning architecture with dependency-aware classification to explicitly model inter-dimensional relationships. Our model achieves average F1 scores of 0.94 and 0.96 on development and test sets, respectively—significantly outperforming strong baselines—and demonstrates robust generalization across both expert-crafted and user-generated counter-speech. This work advances counter-speech evaluation from binary classification toward an interpretable, sociologically informed, and structurally rich paradigm.
This study addresses the limitations of traditional sentiment analysis, which relies solely on customer tone and often fails to accurately capture true satisfaction. The authors propose a novel large language model–based approach that directly extracts structured indicators of customer satisfaction and underlying reasons—such as “tolerated friction”—from over 70,000 customer service dialogues. Leveraging GPT-5.4 for multidimensional annotation and validating labels against actual customer ratings, the method achieves a correlation of 0.47 between predicted satisfaction and ground-truth scores, significantly outperforming conventional sentiment analysis (r = 0.36). It also demonstrates a lower false-positive rate and reveals a notable discrepancy: in 44% of conversations, customer tone diverges from actual satisfaction, underscoring the approach’s superior discriminative capability beyond surface-level sentiment cues.
Recent advances in natural language processing have enabled increasingly accurate estimation of psychological traits from language. However, most existing approaches rely on supervised models trained to predict questionnaire scores, limiting interpretability and generalizability across contexts. The present study introduces a theory-driven and fully unsupervised framework for measuring psychological states directly from natural language using semantic projection. Psychological constructs were operationalized as interpretable semantic axes derived from lexical anchors and items from validated clinical scales assessing depression, anxiety, and worry. Participants textual responses were embedded using Sentence-BERT and projected onto these axes to generate continuous psychological scores across multiple response formats, including selected words, generated words, phrases, and free-text responses. Projection scores were evaluated through correlations with standardized clinical measures , split-half reliability analyses, attenuation corrections, distributional similarity using Wasserstein distance, and comparisons with lexicon-based sentiment analysis (VADER). Results showed strong associations between projection scores and clinical measures, particularly for structured formats such as selected words, written words, and phrases. Free-text responses produced weaker results when analyzed as whole texts, but performance improved substantially when sentence-level aggregation strategies were applied. These findings support semantic projection as an interpretable and scalable alternative to supervised language models for psychological assessment and highlight the importance of response format and text-processing strategies in language-based mental health measurement.
This study addresses the critical gap in culturally aware emotion recognition within conversational AI, particularly the underrepresentation of African Black communities, which undermines ethical performance and system trustworthiness. To bridge this gap, the work proposes a novel multimodal approach that integrates vocal and facial image data, introducing an innovative Audio-Frame Mean Expression (AFME) algorithm. The model employs a three-layer convolutional neural network to simultaneously recognize seven basic emotions and detect sarcasm, explicitly accounting for cultural, regional, and contextual nuances. Experimental results demonstrate strong performance, achieving accuracy rates between 85% and 96% across all tasks, thereby significantly enhancing the adaptability, precision, and reliability of conversational AI systems in this specific cultural context.
Traditional sentiment analysis relies on discrete classification, which struggles to capture the nuanced gradations of emotional intensity required in domains such as finance. This work proposes a novel paradigm that reframes sentiment analysis as a continuous regression task by constructing a dataset annotated with fine-grained emotion intensity scores and fine-tuning open-source generative language models to predict values on a 0–100 scale. The proposed approach significantly outperforms conventional classification baselines and demonstrates strong cross-construct transferability on related affective tasks, including sentiment polarity and arousal. By enabling more precise and expressive modeling of emotional intensity, this method offers enhanced practical utility for real-world applications demanding granular affective understanding.