Unsupervised Multimodal Intent Discovery via MLLM-Guided Concept Generation and Semantic Propagation

2026-07-23

AI Summary

This work addresses the challenges of discovering user intents in unlabeled multimodal dialogues, where the absence of explicit supervision and limited model interpretability hinder performance. The authors propose MCSP, a novel approach that uniquely integrates multimodal large language model (MLLM)-guided contrastive reasoning with semantic propagation. Specifically, high-level semantic concepts generated by an MLLM guide unsupervised clustering, and these semantics are further refined through propagation over a semantically weighted graph to enforce both local consistency and global semantic alignment. Evaluated on three multimodal intent datasets, MCSP significantly outperforms state-of-the-art methods while yielding semantically interpretable clustering outcomes.

Abstract

Unsupervised multimodal intent discovery aims to uncover latent intents from unlabeled multimodal dialogues, but remains challenging due to the lack of explicit semantic supervision. Existing methods often provide limited interpretability, as their refinement mainly relies on geometric similarity rather than high-level semantic guidance. To address these limitations, we propose MCSP, a fully unsupervised method that introduces semantic refinement based on concepts into multimodal intent discovery. To obtain reliable semantic evidence for intent discovery, we identify high-quality representative samples for each cluster and use them to support MLLM-guided contrastive reasoning against neighboring clusters, which produces interpretable high-level semantic concepts. Building on these concepts, we perform semantic propagation over a semantically weighted graph to align conceptual information with local structural consistency and generate reliable pseudo-labels for representation refinement. Extensive experiments on three challenging multimodal intent datasets show that MCSP consistently outperforms state-of-the-art methods while producing interpretable clusters grounded in semantic concepts.

Problem

• unsupervised multimodal intent discovery
• latent intents
• semantic supervision
• interpretability
• multimodal dialogues

Innovation

• unsupervised multimodal intent discovery
• MLLM-guided concept generation
• semantic propagation
• interpretable clustering
• pseudo-label refinement