Hierarchical Dual-Head Model for Suicide Risk Assessment via MentalRoBERTa

2025-10-22

AI Summary

Automated suicide risk detection in social media faces three key challenges: severe class imbalance, complex temporal dynamics in posting sequences, and the dual ordinal–categorical nature of risk levels. To address these, we propose a hierarchical dual-head neural network built upon MentalRoBERTa. Our approach innovatively integrates CORAL ordinal regression with standard classification in a dual-head architecture to explicitly model both the ordered structure and categorical distinctions among risk levels. We incorporate time-interval embeddings and employ a 3-layer Transformer encoder (8 attention heads) to capture temporal dependencies. Additionally, we enhance robustness and efficiency via mixed-precision training, freezing lower-layer parameters, and a joint loss function combining CORAL loss, cross-entropy loss, and Focal Loss. Evaluated on a four-level risk classification task using stratified 5-fold cross-validation, our method achieves a statistically significant improvement in macro-F1 score, effectively mitigating overfitting and overconfidence.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Social media platforms have become important sources for identifying suicide risk, but automated detection systems face multiple challenges including severe class imbalance, temporal complexity in posting patterns, and the dual nature of risk levels as both ordinal and categorical. This paper proposes a hierarchical dual-head neural network based on MentalRoBERTa for suicide risk classification into four levels: indicator, ideation, behavior, and attempt. The model employs two complementary prediction heads operating on a shared sequence representation: a CORAL (Consistent Rank Logits) head that preserves ordinal relationships between risk levels, and a standard classification head that enables flexible categorical distinctions. A 3-layer Transformer encoder with 8-head multi-head attention models temporal dependencies across post sequences, while explicit time interval embeddings capture posting behavior dynamics. The model is trained with a combined loss function (0.5 CORAL + 0.3 Cross-Entropy + 0.2 Focal Loss) that simultaneously addresses ordinal structure preservation, overconfidence reduction, and class imbalance. To improve computational efficiency, we freeze the first 6 layers (50%) of MentalRoBERTa and employ mixed-precision training. The model is evaluated using 5-fold stratified cross-validation with macro F1 score as the primary metric.

Problem

• Classifying suicide risk levels from social media posts
• Addressing class imbalance and temporal posting patterns
• Preserving ordinal relationships while enabling categorical distinctions

Innovation

• Hierarchical dual-head neural network for suicide risk classification
• CORAL head preserves ordinal relationships between risk levels
• Transformer encoder models temporal dependencies in posts