Translating Emotions to Annotations - A Participant Perspective of Physiological Emotion Data Collection

2025-03-25

AI Summary

Physiological emotion data collection suffers from misalignment between subjective labels and objective physiological responses, primarily due to human participant dependency and associated cognitive biases. Method: We conducted a VR-based emotion elicitation study with 37 participants, complemented by semi-structured interviews, to investigate participant-centered factors affecting labeling fidelity. Contribution/Results: We identify three critical human-induced interference factors—perceptual bias, experimental design mismatch, and environmental mismatch—and provide the first systematic characterization of the decoupling mechanism between subjective cognition and physiological response. Based on these findings, we propose a participant-centered experimental design paradigm and a context-enhanced annotation framework, yielding seven actionable guidelines for physiological emotion data collection. This work establishes a human-factor foundation for reliable affective labeling and advances AI-driven affective modeling by bridging cognitive and physiological domains.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Humans and AI: Emotional Intelligence
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Security and Privacy: Large-scale security measurements

Abstract

Physiological signals hold immense potential for ubiquitous emotion monitoring, presenting numerous applications in emotion recognition. However, harnessing this potential is hindered by significant challenges, particularly in the collection of annotations that align with physiological changes since the process hinges heavily on human participants. In this work, we set out to study human participant perspectives in the emotion data collection procedure. We conducted a lab-based emotion data collection study with 37 participants using 360 degree virtual reality video stimulus followed by semi-structured interviews with the study participants. Our findings presented that intrinsic factors like participant perception, experiment design nuances, and experiment setup suitability impact their emotional response and annotation within lab settings. Drawing from our findings and prior research, we propose recommendations for incorporating participant context into annotations and emphasizing participant-centric experiment designs. Furthermore, we explore current emotion data collection practices followed by AI practitioners and offer insights for future contributions leveraging physiological emotion data.

Problem

• Aligning annotations with physiological changes in emotion monitoring
• Understanding participant perspectives in emotion data collection
• Improving emotion annotation accuracy through participant-centric designs

Innovation

• Using 360 VR video for emotion stimulus
• Semi-structured interviews for participant insights
• Participant-centric design for emotion annotation