MENA: Multimodal Epistemic Network Analysis for Visualizing Competencies and Emotions

2025-04-03

AI Summary

This study addresses the need to improve geriatric care quality by investigating the dynamic mechanisms underlying caregivers’ attitudes and emotions during augmented reality (AR) simulation training, particularly how these are modulated by the consciousness level of virtual geriatric patients (VGPs). Method: We propose a multimodal cognitive network analysis framework that integrates eye-tracking, speech, behavioral time-series data, and affective computing. Crucially, we pioneer the incorporation of positive emotion recognition into epistemic network analysis (ENA), enabling coupled modeling of caregiving competence and emotional states, alongside cross-network comparative visualization. Contribution/Results: Empirical findings demonstrate that VGP consciousness significantly enhances caregivers’ supportive behaviors and person-centered engagement. The framework exhibits strong transferability across interpersonal interaction scenarios, offering a novel paradigm for evaluating simulation training efficacy and informing evidence-based intervention design in geriatric care education.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

The need to improve geriatric care quality presents a challenge that requires insights from stakeholders. While simulated trainings can boost competencies, extracting meaningful insights from these practices to enhance simulation effectiveness remains a challenge. In this study, we introduce Multimodal Epistemic Network Analysis (MENA), a novel framework for analyzing caregiver attitudes and emotions in an Augmented Reality setting and exploring how the awareness of a virtual geriatric patient (VGP) impacts these aspects. MENA enhances the capabilities of Epistemic Network Analysis by detecting positive emotions, enabling visualization and analysis of complex relationships between caregiving competencies and emotions in dynamic caregiving practices. The framework provides visual representations that demonstrate how participants provided more supportive care and engaged more effectively in person-centered caregiving with aware VGP. This method could be applicable in any setting that depends on dynamic interpersonal interactions, as it visualizes connections between key elements using network graphs and enables the direct comparison of multiple networks, thereby broadening its implications across various fields.

Problem

• Improving geriatric care quality through stakeholder insights
• Enhancing simulation effectiveness by analyzing caregiver emotions
• Visualizing competencies-emotions links in dynamic care settings

Innovation

• Multimodal Epistemic Network Analysis framework
• Detects positive emotions in caregivers
• Visualizes competencies-emotions via network graphs