DynaBridge: Dynamic Summary-Guided Cross-Task Multimodal Fusion for DASS-Structured Mental Health Assessment

2026-07-28

AI Summary

This work addresses the limitation of existing general-purpose multimodal models in assessing depression, anxiety, and stress (DASS), which often disregard the psychometric structure of the DASS-21 scale and thus fail to properly model the fixed mapping between ordinal symptom items and subscales. To overcome this, the authors propose DynaBridge, a novel framework that explicitly incorporates the DASS-21 structure into multimodal fusion. DynaBridge leverages DASS-aware semantic summaries generated by a frozen large language model as participant-level evidence, integrating acoustic, visual, and textual cues through a dynamic summary-guided cross-task fusion mechanism to predict ordinal item distributions and reconstruct risk evidence. A confidence-aware conservative fusion strategy is further introduced to enhance robustness. Evaluated on the AdoDAS validation set, the method achieves an average F1 score of 0.5012 for D/A/S risk prediction and an average quadratic weighted kappa (QWK) of 0.3216 for item-level prediction, significantly outperforming baseline and state-of-the-art approaches.

Abstract

Multimodal behavioral analysis offers a scalable approach to assessing depression, anxiety, and stress, yet generic fusion models often ignore the psychometric structure of questionnaire labels. In DASS-21, risk labels are derived from ordered symptom items through fixed item-to-subscale mappings. We propose \textbf{DynaBridge}, a dynamic summary-guided cross-task multimodal framework for DASS-structured mental health assessment. DynaBridge encodes acoustic, visual, and textual cues across multiple sessions and augments them with frozen-LLM-generated DASS-aware summaries as participant-level semantic evidence. It predicts ordinal item distributions, reconstructs depression, anxiety, and stress risk evidence from item-level soft scores, and fuses this evidence with direct multimodal risk predictions. A confidence-aware refinement strategy further incorporates high-confidence semantic cues conservatively. On the official AdoDAS validation split, DynaBridge outperforms the official baseline and representative multimodal methods, achieving 0.5012 mean F1 for D/A/S risk prediction and 0.3216 mean QWK for DASS-21 item prediction. These results show the value of bridging multimodal cues, semantic summaries, and DASS-21 psychometric structure.

Problem

• multimodal fusion
• DASS-21
• psychometric structure
• mental health assessment
• ordinal items

Innovation

• dynamic multimodal fusion
• summary-guided learning
• DASS-structured assessment
• ordinal item modeling
• confidence-aware refinement