🤖 AI Summary
This work addresses emotion recognition in conversational scenarios by effectively integrating multimodal information to enhance performance. We propose a lightweight multimodal baseline system that combines a Transformer-based text classifier with a self-supervised speech representation model, employing a simple late-fusion strategy for emotion prediction. Experimental results on the SemEval-2024 Task 3 dataset demonstrate that, under constrained training conditions, our multimodal approach significantly outperforms unimodal models. By providing a transparent and reproducible benchmark system, this study establishes a reliable foundation for future research in multimodal emotion recognition within dialogue contexts.
📝 Abstract
We present a lightweight multimodal baseline for emotion recognition in conversations using the SemEval-2024 Task 3 dataset built from the sitcom Friends. The goal of this report is not to propose a novel state-of-the-art method, but to document an accessible reference implementation that combines (i) a transformer-based text classifier and (ii) a self-supervised speech representation model, with a simple late-fusion ensemble. We report the baseline setup and empirical results obtained under a limited training protocol, highlighting when multimodal fusion improves over unimodal models. This preprint is provided for transparency and to support future, more rigorous comparisons.