MELDAE: A Framework for Micro-Expression Spotting, Detection, and Automatic Evaluation in In-the-Wild Conversational Scenes

2025-10-26

AI Summary

To address the challenges of ambiguous temporal boundaries of micro-expressions and poor generalization of existing models in natural conversational settings, this paper proposes MELDAE, an end-to-end framework. Methodologically, it introduces three key innovations: (1) construction of the first micro-expression dataset specifically curated for natural dialogue; (2) a boundary-aware loss function that explicitly models the temporal structure of micro-expression onset and offset; and (3) joint optimization of micro-expression localization and classification. Evaluated on the WDMD dataset, MELDAE achieves a 17.72% improvement in F₁ᴰᴿ over prior methods. Cross-dataset experiments further demonstrate its superior generalization capability compared to state-of-the-art approaches. This work establishes a scalable, end-to-end solution for micro-expression analysis in realistic conversational scenarios.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Accurately analyzing spontaneous, unconscious micro-expressions is crucial for revealing true human emotions, but this task remains challenging in wild scenarios, such as natural conversation. Existing research largely relies on datasets from controlled laboratory environments, and their performance degrades dramatically in the real world. To address this issue, we propose three contributions: the first micro-expression dataset focused on conversational-in-the-wild scenarios; an end-to-end localization and detection framework, MELDAE; and a novel boundary-aware loss function that improves temporal accuracy by penalizing onset and offset errors. Extensive experiments demonstrate that our framework achieves state-of-the-art results on the WDMD dataset, improving the key F1_{DR} localization metric by 17.72% over the strongest baseline, while also demonstrating excellent generalization capabilities on existing benchmarks.

Problem

• Spots and detects micro-expressions in natural conversational environments
• Addresses performance degradation in real-world scenarios versus lab settings
• Improves temporal accuracy of micro-expression onset and offset localization

Innovation

• Dataset for conversational wild micro-expressions
• End-to-end localization and detection framework
• Boundary-aware loss improves temporal accuracy