From Sparse Representations to Behavioral Insights for Multimodal Depression Assessment

2026-10-08

AI Summary

This study addresses the challenges of dense, uninterpretable representations in multimodal depression assessment by proposing BehavDep, a novel framework that introduces sparse factorization and semantic bridging mechanisms to disentangle multimodal behaviors into sparse latent factors linked to interpretable concepts. By integrating weakly supervised learning with multi-view aggregation, the framework enables precise evaluation from video-level to user-level assessments. Experimental results demonstrate that BehavDep achieves state-of-the-art overall performance while effectively revealing modality complementarity and heterogeneous behavioral patterns. Furthermore, it supports concept-level analysis of editing responses, facilitating a paradigm shift from black-box prediction to structured behavioral insight.

Abstract

Multimodal depression assessment offers a promising approach to analyzing behavioral patterns associated with depression. However, existing methods often rely on dense and opaque multimodal representations, making it difficult to interpret the behavioral patterns underlying their predictions. In this work, we introduce BehavDep, a sparse factor-based framework that decomposes multimodal behavioral representations into sparse latent factors and associates them with behaviorally meaningful concepts through a semantic bridge. To address the mismatch between user-level annotations and heterogeneous video-level behaviors, BehavDep further learns video-level depression tendency scores under weak supervision and aggregates information across multiple observations for user-level assessment. Extensive experiments demonstrate that BehavDep achieves the best overall assessment performance while revealing complementary modality contributions, heterogeneous behavioral patterns across observations, and prediction responses to concept-level editing. These results show that BehavDep provides a structured and interpretable approach to analyzing multimodal behavioral representations for depression assessment.

Problem

• multimodal depression assessment
• interpretability
• sparse representations
• weak supervision
• behavioral patterns

Innovation

• Sparse Representations
• Multimodal Depression Assessment
• Interpretability
• Weak Supervision
• Semantic Bridge