Problem
Research questions and friction points this paper is trying to address.
Intercultural Communication
Language Learning Agent
Nonverbal Behavior
Social Norm Violations
Cultural Adaptation
Toward a Culturally Adapted Chinese Language Agent: A Wizard-of-Oz Study of Nonverbal Behavior in Chinese-German Intercultural Interaction
🤖 AI Summary
This study addresses the insufficient support for cultural norm sensitivity in Chinese language learning among native German speakers by developing a real-time multimodal interaction system. Methodologically, it employs a Wizard-of-Oz paradigm integrating MetaHuman photorealistic avatars, Live Link facial capture, and MediaPipe pose tracking to collect behavioral data. Furthermore, it proposes a psychology-grounded hierarchical annotation framework encompassing implicit socio-emotional responses and cultural norm interpretation, thereby enabling high-ecological-validity cross-cultural scenario design. Ultimately, this project establishes a multimodal corpus that provides the technical and methodological foundation for developing culturally adaptive intelligent tutoring agents and training automated models for cultural explanation and behavior generation.
📝 Abstract
Successful intercultural communication requires more than grammatical competence. It demands sensitivity to culturally embedded social norms whose violation triggers subtle but meaningful nonverbal responses. For German learners of Mandarin Chinese, acquiring this sensitivity is critical yet poorly supported by existing language-learning agents. We present a Wizard-of-Oz (WoZ) study design and supporting real-time system for collecting multimodal behavioral data from native Chinese speakers reacting to social norm violations by German learners. The system features a photorealistic MetaHuman avatar driven by Live Link face capture and MediaPipe upper-body tracking, a wizard console for real-time behavior selection, and synchronized multimodal logging across agent and learner streams. A layered annotation framework, based on psychological theory and covering non-observable socioemotional reactions, norm interpretation, verbal, and observable behavior thereof, and future supervision targets enables the corpus to support training of future automated cultural interpretation and behavior generation models. Four ecologically valid interaction scenarios, developed with cultural and pedagogical experts, provide the methodological and technical foundation for a culturally adapted conversational agent for Chinese language learning.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Wizard-of-Oz study
multimodal data collection
MetaHuman avatar
layered annotation framework
culturally adapted agent
🔎 Similar Papers
No similar papers found.
